test: benchmark DP2 TP32 with DFlash

README.md

@@ -1,5 +1,39 @@
 # sskj — 多平台大模型推理性能基准测试项目
 
+**更新（2026-08-31 18:22:43 CST）**：完成 Kimi-K3 PP8 + DFlash 八节点 PD 部署验证。P 601-604 为 TP4/PP8/EP4，D 605-608 为 TP32/PP1/EP4，均使用 FlashInfer MXFP4、BF16 KV、8K Chunk。固定 GSM8K 64 题 C1/C8 共 128/128 请求成功，答案检查全部通过且无输出截断；按 verify 步数加权的接受长度为 6.4211/6.5158，整体 Output TPS 为 33.38/117.56。修复评测容器的数据集路径，保留原失败与续跑证据；结果、完整服务日志及资源清理记录已归档。历史 5-shot 与前 5 题重叠，本轮作为部署回归而非独立准确率评测。详见 [实现与结果](experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md)。
+
+**更新（2026-08-31 17:50:51 CST）**：修复 Kimi 混合注意力缓存池在 PD 初始化时缺少 `end_layer` 的兼容问题，按完整注意力层偏移和本地层数确定传输范围，保留普通池与 layer-shard 路径。新增回归后 24 项 CPU 测试通过，八节点修复镜像摘要一致；P 组已完成预热和 Mooncake 注册，健康检查返回 200，D 组正在验证启动。当前 Run 为 `pd-dflash-kvbounds1-20260831-1740`，尚无 GSM8K 验收结果。详见 `experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md`。
+
+**更新（2026-08-31 17:05:00 CST）**：新增 Kimi-K3 PP8 + DFlash PD 适配验证入口。基于 SGLang PR #33863 固定源码，接通 PP 分段 hidden 投影、P 侧 prompt draft KV 生成、D 侧输入生命周期与 TP4→TP32 的 draft GQA KV 传输，保留 Kimi SM120 FlashInfer MXFP4 接入。修复 P 普通预热误带 DFlash verify metadata 的启动问题，23 项 CPU 回归通过；修复镜像已同步 601–608，服务级验证正在进行，尚无 GSM8K 结果。配置为 P TP4/PP8/EP4、D TP32/PP1/EP4、BF16 KV、8K Chunk，计划固定 64 题 C1/C8。详见 `experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md`。
+
+**更新（2026-08-27 13:53:26 CST）**：完成 Kimi-K3 八节点标准 PD 第一阶段。P 组 601-604 使用 PP8×TP4×EP4、FlashInfer MXFP4、Chunk 8K，D 组 605-608 使用 PP1×TP32×EP32、Marlin，通过 Mooncake 0.3.12.post1 和 4 Rail RDMA 传输；统一 P/D `page_size=64` 后，16K→1 与 16K→512 的 C1/C8 共 91/91 请求成功。代表结果：16K→1 C8 Input TPS 6364.31、TTFT P50/P95 20.555/21.345 秒；16K→512 C8 TPOT P50/P95 63.20/66.55 ms。详见 `experiments/pro6000/kimi3_pro6000_pd_pp8_standard/README.md`。
+
+**更新（2026-08-20 10:13:01 CST）**：完成 Kimi-K3 Prefill TP Reduce Scatter 可行性审计并停止该方向。K3 的 MLA 输出门控仍依赖完整 7168 维 hidden，且 69/93 层为 KDA；保留或恢复 gate hidden 后，原 hidden All-Reduce 无法消除并新增 2112 维 latent All-Gather，估算通信量反增约 14.7%。研究原型仅保留为否决证据，不进入四节点实验或上游 PR；后续转向 MoE A2A 与 Pipeline Parallelism。详见 `experiments/pro6000/kimi3_pro6000_sglang_tp_reduce_scatter_prefill/README.md`。
+
+**更新（2026-08-19 16:51:43 CST）**：完成 Kimi-K3 / RTX 6000D / SGLang Prefill MoE backend 交付文档收敛与 Chunk=4K 补测。四节点 TP32×EP4、16K→1、C=8/16 下 12/12 重复、480/480 请求成功；FlashInfer 相对 Marlin 提升约 9.8% Input TPS、降低约 9.0% TTFT。结合 4K/8K/16K 全部结果，最终推荐保持 `flashinfer_mxfp4 + chunked_prefill_size=8192`。
+
+**更新（2026-08-19 14:23:52 CST）**：601 的 GitLab SSH key 已生效，内部 Kimi-K3 SM120 审计分支已推送至 `qqtang/sskj:hzy-kimi-k3-sm120-flashinfer-mxfp4`。远端已有普通分支 `hzy`，因此不能创建 `hzy/...` ref，最终采用无层级冲突的分支名；GitHub SGLang feature 分支保持不变。
+
+**更新（2026-08-19 13:58:50 CST）**：用户同步 GitHub fork 后，将 Kimi-K3 SM120 SGLang 的 7 个提交无冲突重放到最新 `main` `593b1a9`，`range-diff` 逐提交确认补丁等价；最终 Draft HEAD `b95b534` 已推送至 `ACEEE-1222/sglang:hzy/kimi-k3-sm120-flashinfer-mxfp4`。在 601 统一验证镜像中重跑最终 HEAD 的语法与定向回归，结果为 `10 passed, 3 deselected`；代表性四机性能数据保留，不重复消耗 GPU。
+
+**更新（2026-08-19 13:07:48 CST）**：完成 Kimi-K3 SM120 SGLang Draft PR 的最终代表点验收。基于 SGLang `300c87a` 与 FlashInfer #4460 `b525c51`，在 601-604 四节点 TP32×EP4 上完成 16K→1、C=8、Chunk=8K 的 Marlin/FlashInfer 各 3 次 A/B；240/240 请求成功。FlashInfer 的 Input TPS 中位数提升 10.96%，TTFT P50/P95 分别下降 9.91%/9.92%。公开 PR Checklist 已全部勾选，轻量汇总和原始证据索引已归档。
+
+**更新（2026-08-19 10:45:18 CST）**：新增 Kimi-K3 SM120 SGLang Draft PR 的单点代表性验收入口。使用精确 SGLang `300c87a`、FlashInfer #4460 实现 `b525c51` 和统一镜像，在 601-604 四节点 TP32×EP4 上仅复测 16K→1、C=8、Chunk=8K 的 Marlin/FlashInfer A/B，各 3 次重复；完整命令、镜像身份、原始日志和汇总统一落入单个 Run 目录。
+
+**更新（2026-08-18 23:00:42 CST）**：完成依赖 FlashInfer #4460 的 SGLang Draft PR 收敛。确认不提交任何 FlashInfer PR，也不 vendor 或 pin 未合并 kernel；在 601 GPU6 上从 #4460 源码构建 FlashInfer 0.6.18 wheel，SGLang 定向测试 8/8 通过。Draft 标题、依赖、兼容策略、测试和四机 EP4 数据见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/SGLANG_DRAFT_PR.md`。
+
+**更新（2026-08-18 22:21:40 CST）**：完成 Kimi-K3 SM120 FlashInfer MXFP4 去重审计。确认 CUTLASS SiTU kernel 已由未合并的 FlashInfer PR #4460 实现，不再提交平行 kernel；SGLang 贡献收敛为 Kimi gate/up 与 scale 布局、SiTU 4.0/25.0 参数映射、非连续输入和 SM120 attention-residual guard，并保留 601-604 EP4 的全部服务级结果。详见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/UPSTREAM_DUPLICATION_AUDIT.md`。
+
+**更新（2026-08-18 18:38:17 CST）**：完成 Kimi-K3 四节点 TP32×EP4 的真实 Prefill MoE backend 验收；24/24 重复均 40/40 成功，FlashInfer MXFP4 在 8K Chunk、C=8/16 下相对 Marlin 提升约 11% Input TPS、降低约 10% TTFT，并确认 8K 优于 16K。实验默认 EP 改为 4，补齐非连续 Prefill 输入与 FlashInfer 官方 SiTU 回归，详见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/README.md`。
+
+**更新（2026-08-18 13:55:30 CST）**：Kimi-K3 四节点 MoE backend 验收入口支持通过环境变量覆盖 EP size 及 backend/chunk/concurrency 子矩阵，用于先执行 TP32×EP4、16K Chunk、16K→1、C=16 的最大压力容量验证，同时保留默认 TP32×EP32 全矩阵。
+
+**更新（2026-08-18 13:03:33 CST）**：Phase 5 候选镜像改为针对 Kimi 基础镜像原生 SGLang 源码应用最小兼容补丁，仅补齐 SM120 FlashInfer MXFP4 backend 分发、权重布局与 SiTU 激活映射，避免整体替换新版 Python 包造成 `sglang-kernel` 版本错配。
+
+**更新（2026-08-18 12:24:19 CST）**：新增 Kimi-K3 / 601-604 / SGLang TP32×EP32 的真实 Prefill MoE backend 验收实验；固定比较 Marlin 与 FlashInfer MXFP4 在 16K→1、C=8/16、Chunk=8K/16K 下的 TTFT、E2E 与 Input TPS，并提供四节点一致候选镜像构建、原始证据和自动汇总，详见 `experiments/pro6000/kimi3_pro6000_sglang_tp32ep32_moe_backend_prefill/README.md`。
+
+**更新（2026-08-17 11:56:47 CST）**：完成 Kimi-K3 / RTX PRO 6000D / SGLang FlashInfer MXFP4 MoE Phase 2 correctness harness；SwiGLU 控制矩阵 8/8 通过，覆盖 edge routing、EP global ID、NaN/Inf、确定性、CUDA Graph 和 Kimi 真尺寸，SiTU 目标矩阵 8/8 稳定复现同一 C++ 缺口，详见 `experiments/pro6000/kimi3_pro6000_sglang_sm120_flashinfer_mxfp4/README.md`。
+
 > 历史更新见 `git log`。项目目的与工作流见下方。
 > **项目目的**：当新显卡（GPU/NPU）到货时，用最短时间完成大模型在该平台上的推理性能评估与部署配置选型。
 > 当前模型：DeepSeek-V4-Flash（FP8 / INT8）；后续接入 GLM5.2，**完全复用**本项目的实验与报告流程。
@@ -45,6 +79,9 @@
 | `experiments/pro6000/dsv4_pro6000_vllm_tp_dp_matrix/` | RTX 6000D + vLLM |
 | `experiments/pro6000/dsv4_pro6000_sglang_tp_dp_matrix/` | RTX 6000D + SGLang |
 | `experiments/pro6000/kimi3_pro6000_sglang_tp32ep32/` | RTX 6000D + SGLang，Kimi-K3（TP32×EP32，部署手册见 docs/KIMI_K3_DEPLOY.md） |
+| `experiments/pro6000/kimi3_pro6000_sglang_tp32ep32_moe_backend_prefill/` | RTX 6000D + SGLang，Kimi-K3 真实 Prefill 的 Marlin/FlashInfer MXFP4 与 Chunk 8K/16K 对照 |
+| `experiments/pro6000/kimi3_pro6000_sglang_tp_reduce_scatter_prefill/` | Kimi-K3 Prefill TP Reduce Scatter 可行性审计；因 MLA gate 与 KDA 依赖完整 hidden，结论为不推荐继续 |
+| `experiments/pro6000/kimi3_pro6000_pd_pp8_standard/` | Kimi-K3 八节点标准 PD：P=PP8/TP4/EP4、D=PP1/TP32/EP32、Mooncake 4 Rail RDMA，含 16K→1/512 C1/C8 原始证据 |
 | `experiments/pro6000/kimi3_pro6000_pd_rdma/` | RTX 6000D + SGLang，Kimi-K3 PD 分离（MoonCake RDMA，8 节点，见 deploy_pd.sh + docs/KIMI_K3_DEPLOY.md 附录 B） |
 
 每个目录内：`run_bench.sh` 跑固定并发矩阵；`run_adaptive_concurrency.sh` 从 C=1 指数倍增搜饱和点；`run_adaptive_concurrency_add16.sh` 从 C=16 线性 +16 步进、带 TTFT SLO 停止与回退（当前主力用法，见 `experiments/ADAPTIVE_CONCURRENCY_USAGE.md`）。

docs/Kimi-K3_DP2_TP32_PP8_16K512_C8对比.md

@@ -0,0 +1,85 @@
+# Kimi-K3 DP2 无投机：TP32 与 PP8 对比
+
+## 测试目的
+
+在 8 台 RTX 6000D 上部署两个独立四节点 SGLang 副本，通过 SGLang Router `round_robin` 组成 DP2，比较两种单副本并行策略：
+
+- TP32：`TP32/PP1/EP4`
+- PP8：`TP4/PP8/EP4`
+
+本轮不启用投机解码，重点观察长输入场景下 TTFT 与长 Decode 的取舍。
+
+## 固定口径
+
+| 项目 | 配置 |
+|---|---|
+| 数据集 | SPEED-Bench `/data/speedbench16k_complete.jsonl` |
+| 请求 | 40 条，16K 输入，512 输出 |
+| 客户端并发 | C=8 |
+| Chunked Prefill | 8K |
+| MoE | EP4，FlashInfer MXFP4，A2A=none |
+| KV Cache | BF16 |
+| Prefix Cache | 关闭 |
+| Router | 两副本 `round_robin` |
+| GPU | 601-608，共 64 卡 |
+
+两种配置均在正式计时前分别预热两个副本，并由 benchmark 再执行一条标准 warmup。`Peak concurrent requests=12` 是 SGLang 报告的服务端峰值，可因排队高于客户端 `max_concurrency=8`，不表示客户端发出了 12 路并发。
+
+## 结果
+
+| 指标 | DP2 TP32/PP1/EP4 | DP2 TP4/PP8/EP4 | PP8 相对 TP32 |
+|---|---:|---:|---:|
+| 成功请求 | 40/40 | 40/40 | 相同 |
+| Benchmark duration | 238.18s | 370.30s | +55.47% |
+| Input TPS | 2797.64 | 1799.46 | -35.68% |
+| Output TPS | 85.99 | 55.31 | -35.68% |
+| Total TPS | 2883.63 | 1854.76 | -35.68% |
+| TTFT mean | 8.35s | 7.84s | -6.15% |
+| TTFT median | 5.30s | 7.03s | +32.62% |
+| TTFT p95 | 20.79s | 11.68s | -43.82% |
+| TPOT mean | 75.93ms | 126.86ms | +67.07% |
+| TPOT median | 79.95ms | 136.09ms | +70.22% |
+| TPOT p95 | 88.06ms | 145.13ms | +64.80% |
+| E2E mean | 47.15s | 72.66s | +54.10% |
+| E2E p95 | 59.24s | 82.83s | +39.82% |
+
+## 初步结论
+
+PP8 继续体现了 Prefill 优势：平均 TTFT 小幅下降 6.15%，TTFT p95 降低 43.82%，长尾更稳定。这来自 TP 通信域从 32 缩到 4，并由流水线分摊模型层。
+
+但 512-token Decode 暴露了 PP8 的代价。每个 token 都要依次通过 8 个 Pipeline Stage，TPOT 增加 67.07%，最终使 Output TPS 降低 35.68%、E2E 平均时延增加 54.10%。因此：
+
+- 以 P 节点 TTFT 为目标时，PP8 有价值，尤其能改善 TTFT 长尾。
+- 作为不做 PD 分离的完整 16K→512 服务，TP32 综合表现更好。
+- PP8 更适合作为 PD 分离中的 P 侧配置，由独立 D 侧承担长 Decode；不能用本轮 PP8 的 E2E/TPOT 直接否定其 P 侧价值。
+
+TP32 的 TTFT median 更低但 p95 明显更高，说明该配置在 DP2 排队与长 Prefill 交错时波动更大。后续如继续优化单实例，应分别报告 median 与 p95，不能只看 mean。
+
+## 数据质量与日志说明
+
+两轮 Router 初始命令没有挂载模型目录，因此 worker tokenizer 注册出现错误。Router 使用固定 `round_robin`，不依赖 tokenizer 做 cache-aware 路由；40/40 请求成功，两副本 Prefill 数量接近均分，未发现请求级失败或重试，所以不影响本轮 A/B 结论。脚本已补上 Router 模型只读挂载，后续运行不会再产生该注册错误。
+
+满负载期间 Router 偶有 `/health` 超时，这是模型服务忙于长 Prefill/Decode 时健康响应延迟；两轮均未出现 OOM、NCCL collective failure、Traceback 或 EngineDeadError。
+
+## 原始结果
+
+TP32：
+
+```text
+/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/
+```
+
+PP8：
+
+```text
+/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/
+```
+
+核心 benchmark 日志分别位于：
+
+```text
+bench/tp32_speedbench16k_c8_o512.log
+bench/pp8_speedbench16k_c8_o512.log
+```
+
+每个结果目录同时保存完整启动命令、环境 manifest、输入校验值、Router 日志和八节点服务日志。

docs/Kimi-K3_PD-DFlash_16K512_C8性能归因.md

@@ -0,0 +1,229 @@
+# Kimi-K3 PD+DFlash 16K→512、C=8 性能归因
+
+更新时间：2026-09-02
+
+## 1. 目标与结论
+
+本实验在 8 台 RTX 6000D 上，用同一镜像、同一 SPEED-Bench 请求和同一服务参数，对比：
+
+- 标准 PD，无投机解码；
+- PD+DFlash；
+- 历史 PP8 双实例无投机结果，用作同数据集拓扑参考。
+
+核心结论：**当前 PD+DFlash 同时放大了 P→D 交接成本和 D 侧单步成本，平均接受长度 3.69 不足以覆盖这些开销。瓶颈不是 400G Rail 被打满。**
+
+与严格同口径 PD 无投机相比，PD+DFlash：
+
+- P→D 每请求交接数据约从 `85.76 MB` 增至 `183.54 MB`，增加 `114.0%`；
+- D 侧 transfer wait 窗口从 `15.24 s` 增至 `22.24 s`，增加 `7.00 s`；
+- Mean TTFT 增加 `7.40 s`，其中 transfer wait 增量相当于 TTFT 增量的 `94.5%`；
+- Mean TPOT 增加 `41.9%`，Input/Output TPS 均下降 `29.0%`；
+- Nsight 中 `_fwd_kernel` 占 GPU kernel 时间 `49.8%`，NCCL AllReduce 占 `31.9%`，两者远高于 MoE/GEMM。
+
+这里的 `transfer wait` 是 D 请求进入后等待 P 完成计算、bootstrap/注册并交付数据的整段窗口，不等于纯 RDMA wire time。
+
+## 2. 指标口径
+
+设第 `i` 个请求的发出、首 token、结束时刻为 `s_i/f_i/e_i`，输入和输出 token 数为 `I_i/O_i`，整轮 benchmark 墙钟时间为 `D`：
+
+| 指标 | 计算方式 | PD 场景中的含义 |
+|---|---|---|
+| TTFT | `f_i - s_i` | Router、P 排队/Prefill、P→D 交接、D 接入及首 token 的总和 |
+| TPOT | `(e_i - f_i) / (O_i - 1)` | 首 token 后平均每个输出 token 的耗时，主要由 D 侧决定 |
+| E2E | `e_i - s_i` | 完整请求延迟 |
+| Input TPS | `ΣI_i / D` | 整个系统墙钟吞吐，不是 P 的纯 Prefill 算力 |
+| Output TPS | `ΣO_i / D` | 整个系统墙钟吞吐，不等于 `1 / Mean TPOT` |
+
+DP、Router 和 PD 不改变公式，只改变时间花在哪个阶段。Profiler 本身开销很大，因此性能数字只取无 profiler 的严格 A/B；Nsight/PyTorch Trace 只用于归因。
+
+## 3. 严格 A/B 配置
+
+| 项目 | P 组 | D 组 |
+|---|---|---|
+| 节点 | 601-604 | 605-608 |
+| 并行 | TP4 / PP8 / EP4 | TP32 / PP1 / EP4 |
+| MoE | FlashInfer MXFP4，A2A none | FlashInfer MXFP4，A2A none |
+| Chunk | 8192 | 8192 |
+| KV Cache | BF16 | BF16 |
+| 请求上限 / CUDA Graph BS | 8 / 8 | 8 / 8 |
+| 显存比例 | 0.84 | 0.86 |
+| 传输 | Mooncake，mlx5_0/1/2/3 | Mooncake，mlx5_0/1/2/3 |
+
+请求固定为同一批 SPEED-Bench 数据：40 请求，C=8，输出 512，实际输入共 666,343 token，warmup=1，seed=42，temperature=0。DFlash 的 draft token 数为 16。
+
+镜像：
+
+```text
+local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
+```
+
+## 4. 端到端结果
+
+| 指标 | PP8 双实例无投机（历史参考） | 严格 PD 无投机 | 严格 PD+DFlash | DFlash vs PD 无投机 |
+|---|---:|---:|---:|---:|
+| 成功请求 | 40 | 40 | 40 | - |
+| Benchmark duration | 335.95 s | 255.29 s | 359.53 s | +40.8% |
+| Input TPS | 1983.46 | **2610.18** | 1853.35 | -29.0% |
+| Output TPS | 60.96 | **80.22** | 56.96 | -29.0% |
+| Mean TTFT | **6.88 s** | 15.40 s | 22.80 s | +48.1% |
+| Median TTFT | **7.04 s** | 16.03 s | 23.07 s | +43.9% |
+| P95 TTFT | **10.64 s** | 22.18 s | 40.54 s | +82.8% |
+| Mean TPOT | 116.13 ms | **63.22 ms** | 89.70 ms | +41.9% |
+| P95 TPOT | 136.01 ms | **70.45 ms** | 144.76 ms | +105.5% |
+| Mean E2E | 66.22 s | **47.70 s** | 68.63 s | +43.9% |
+| DFlash 接受长度 | - | - | 3.69 | - |
+
+PP8 双实例结果使用相同数据，但缺少本轮完整命令归档，只作为拓扑参考。严格单变量结论来自后两列。
+
+## 5. TTFT 瀑布拆解
+
+从 P 最后一个 PP stage（PP7/TP0）和 D TP0 日志中去重，排除 warmup 后取正式 40 请求：
+
+| 阶段 | PD 无投机 | PD+DFlash | 变化 |
+|---|---:|---:|---:|
+| P bootstrap | 5.18 s | 6.67 s | +28.7% |
+| P forward | 9.51 s | 14.22 s | +49.5% |
+| P queue | 1.94 s | 0.93 s | -52.3% |
+| P→D payload / 请求 | 85.76 MB | 183.54 MB | **+114.0%** |
+| D bootstrap | 5.15 ms | 4.85 ms | 基本不变 |
+| D allocation wait | 12.67 ms | 2.78 ms | 可忽略 |
+| D queue | 0.41 ms | 0.30 ms | 可忽略 |
+| D transfer wait | 15.24 s | 22.24 s | **+45.9%** |
+| D forward | 32.35 s | 46.10 s | **+42.5%** |
+
+### 5.1 为什么说 PD 交接冲散了收益
+
+DFlash 需要向 D 节点交接额外的 draft/KDA 状态，使每请求 payload 增加约 `97.78 MB`。D 侧 transfer wait 随之增加 `6.998 s`，而 Mean TTFT 总共增加 `7.402 s`。两者不是可相加的独立阶段，但增量高度一致，说明 TTFT 回退主要发生在 P 计算与 P→D 交接链路，而不是 D admission 排队。
+
+### 5.2 为什么不能把 22.24 秒叫作 RDMA 拷贝耗时
+
+该时间戳从 D 开始等待传输到请求可进入 forward，覆盖：
+
+```text
+P 侧计算/准备 → Mooncake bootstrap 与 buffer 注册 → 数据传输 → D 侧完成可用
+```
+
+因此它是协议端到端等待窗口。纯线速传输 `183.54 MB` 即使只用一条 400G Rail，理论时间也只有毫秒级；当前几十秒来自依赖链、同步与大量小通信，而不是字节在线上连续搬运了几十秒。
+
+## 6. RDMA 与 GPU 证据
+
+严格 DFlash run 的 D 节点活跃 Rail：
+
+- `mlx5_0` RX P95 约 `39.25-39.38 Gbit/s`；
+- `mlx5_2` TX P95 约 `39.14-39.23 Gbit/s`；
+- 峰值约 `42 Gbit/s`；
+- 单 Rail 标称为 400G，当前峰值约为其 10%。
+
+所以当前不是 raw bandwidth saturation。需要优化的是交接字节量、通信调用方式、同步依赖和通信域，而不是先增加 Rail 带宽。
+
+严格 A/B 中 D 节点 GPU Util P95 均为 100%，DFlash D 节点平均利用率约 74%，并非 GPU 长期空闲等待网络；但高利用率不能区分有效计算、通信 kernel 和等待慢 rank，需结合 Nsight。
+
+## 7. Nsight Systems 归因
+
+Nsight 只包裹 D0 节点的 8 个本地 scheduler，保持 CUDA Graph 开启。Profile run 为 8 请求，仅用于时间线归因；其吞吐和延迟不能与无 profiler 结果比较。
+
+### 7.1 全局 GPU kernel
+
+| Kernel 类别 | GPU 时间 | 占比 | 调用数 |
+|---|---:|---:|---:|
+| `_fwd_kernel` | 14.59 s | **49.8%** | 4,590 |
+| NCCL AllReduce Ring LL BF16 | 9.34 s | **31.9%** | 30,600 |
+| MoE/GEMM 主 kernel | 约 1.0 s | 约 3.4% | - |
+| NCCL AllGather | 0.53 s | 1.8% | - |
+| KDA update | 0.42 s | 1.4% | - |
+
+`_fwd_kernel` 通过 NVTX 相关性确定处于 DFlash target verify/draft forward 路径，但仅凭导出的 kernel 名还不能把它进一步等同为某个单一源码函数。
+
+### 7.2 DFlash NVTX 阶段与 kernel
+
+本轮额外标记：draft prepare、target verify、accept、ReplaySSM commit 和 draft KV materialize。
+
+| NVTX 阶段 | 主要 GPU 内容 |
+|---|---|
+| Target verify | `_fwd_kernel` 51.0%；AllReduce 30.6%；MoE/GEMM 13.0%；AllGather 1.8% |
+| Draft prepare/forward/sample | AllReduce 66.8%；`_fwd_kernel` 25.4%；MoE/GEMM 4.0%；AllGather 2.3% |
+| Materialize draft KV | 主要为 GEMM/数据变换，GPU 总量较小 |
+| ReplaySSM commit / accept | GPU 总量很小，不是主瓶颈 |
+
+NVTX host range 平均每个 speculative cycle：
+
+- target verify：约 18.07 ms；
+- draft prepare/forward/sample：约 9.26 ms；
+- materialize draft KV：约 1.44 ms；
+- ReplaySSM commit：约 0.52 ms；
+- accept：约 0.43 ms。
+
+CUDA API 侧 `cudaEventSynchronize` 共约 `24.34 s`，占 CUDA API 时间 `82.9%`；`cudaGraphLaunch` 320 次、约 `2.89 s`。这说明 CUDA Graph 已生效，但每轮 speculative cycle 仍暴露出 target/draft/NCCL 依赖的同步等待。Graph 能减少 launch overhead，不能消除 collective 和状态依赖。
+
+### 7.3 PyTorch Profiler 补充结果
+
+PyTorch Profiler 已导出 P0 本地 PP0/PP1 的 8 份 trace，共约 `117 MiB`。但 P/D 的 `/stop_profile` 均返回 `Internal Server Error`，D trace 未落盘；profile 期间客户端流式 TTFT 也失真。因此本报告不从这轮生成新的性能结论，保留 P trace 供后续定位 Prefill operator stack。
+
+实验入口已修正：P/D 的 `stop_profile` 改为并行请求并各自限时 180 秒，避免一侧导出异常让整轮实验永久挂起。
+
+## 8. 两个问题的回答
+
+### 8.1 PD 的收益为什么被传输冲散
+
+不是单纯“网卡慢”，而是 DFlash 改变了 PD 协议要交接的内容：payload 翻倍、P 侧准备时间增加、D 侧必须等更多状态就绪。当前 D 侧 Rail 远未饱和，因此提升网卡带宽不会按比例降低 7 秒差额。
+
+### 8.2 投机解码为什么收益很少，甚至变慢
+
+平均接受长度 3.69 意味着一次 draft/verify 只推进约 3.69 个 token。每轮仍需：
+
+```text
+draft forward/sample
+→ target verify
+→ 大量 TP32 AllReduce
+→ KDA/ReplaySSM 状态处理
+→ accept 与 KV materialize
+```
+
+Nsight 显示 target verify 和 AllReduce 已占关键路径。接受长度不足以摊薄这些成本，最终 D forward 增加 42.5%，TPOT 增加 41.9%。
+
+## 9. 后续优化顺序
+
+1. **先减少 PD 交接内容**：确认哪些 draft/KDA buffer 必须由 P 传输，哪些可以由 D 本地重建或延迟生成；用 payload/request 和 TTFT waterfall 验收。
+2. **降低 D 侧通信固定成本**：针对 TP32 小消息 AllReduce 比较更小通信域或 per-op Tree；必须保持同一 SPEED-Bench 复测正确性、TPOT 与接受长度。
+3. **搜索 DFlash 的收益边界**：固定 PD 拓扑，扫 draft token 数和 tree 参数，目标是提高接受长度同时减少 verify 次数；不要只看接受长度，要看 `accepted tokens / speculative cycle cost`。
+4. **最后再调 Mooncake/RDMA**：只有 HCA 接近线速或纯传输微基准异常时，才把 Rail 带宽作为主优化项。
+
+## 10. 原始证据
+
+严格 PD 无投机：
+
+```text
+/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-nospec-ab-pmem084-20260901-1808/
+```
+
+严格 PD+DFlash：
+
+```text
+/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/
+```
+
+Nsight：
+
+```text
+/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/
+/data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/profiles/node_5/nsys/d0.nsys-rep
+```
+
+PyTorch Profiler（仅 P0 的 PP0/PP1 trace 有效）：
+
+```text
+/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-torch-retry-20260902-103042/torch/
+```
+
+PP8 双实例历史参考：
+
+```text
+/data/cdflash_speedbench/matrix/bench_dual_16k_o512_c8.log
+/data/cdflash_speedbench/matrix/bench_dual_16k_o512_c8.jsonl
+```
+
+本地归档：
+
+```text
+/Users/hzy/Desktop/infra/docs/evidence/kimi_k3_pd_dflash_attribution/
+```

docs/Kimi-K3_PP与DFlash迁移审计.md

@@ -0,0 +1,119 @@
+# Kimi-K3：从 PP + DSpark 迁移到 PP + DFlash
+
+更新：2026-08-31；PP8 + PD + DFlash 集成与 GSM8K 64 题 C1/C8 验证完成。
+
+## 1. 决策与范围
+
+**优先推进 DFlash 的 PP + PD 适配。** #33863 的分段投影设计能复用于 DFlash，但需要补齐 DFlash worker、调度入口和异构 TP 下的 draft KV 传输。
+
+已实现 DFlash PP worker、Kimi 跨 stage capture、PD 输入衔接、异构 TP 的 draft KV 传输及 Kimi FlashInfer 布局/SiTU 接入。24 项 CPU 回归及 SM120 GPU 集成回归通过。Run `pd-dflash-kvbounds1-20260831-1740` 已完成跨组 smoke 与 GSM8K 64 题 C1/C8：两轮各 64/64 请求和答案检查通过，均无输出截断；按验证步数加权的接受长度为 6.4211/6.5158，整体 Output TPS 为 33.38/117.56。八节点实验容器已清理、GPU 已释放。详见 [实现、结果与实际命令](../experiments/pro6000/kimi3_pro6000_pd_dflash_validation/README.md)。
+
+暂停 EAGLE3 baseline。KV cache 保持 BF16，chunk 保持 8192。FlashInfer 使用官方 0.6.18 加已合并 #4460 的显式 backport；原版 0.6.18 wheel 尚未包含所需 CUTLASS SiTU 接口。
+
+已验证拓扑：P 使用 601–604 的 TP4/PP8/EP4；D 使用 605–608 的 TP32/PP1/EP4，两组均为 FlashInfer MXFP4。这里适配的是 **P 侧 PP 生成 draft 上下文**，D 侧投机执行仍为 PP1。
+
+前期源码审计快照（PR 状态为该次审计记录，不表示实时状态）：
+
+| 对象 | 版本 |
+|---|---|
+| SGLang main | `3139ceaeec50868a441d68dc231663f3777e0d93` |
+| PR #33863 head | `6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5` |
+| PR 状态 | Open；维护者要求拆分 DSv4 PP/PD、DSpark、Kimi/线性注意力工作 |
+
+PR 公布的准确性数据来自 DSv4 Flash / H20，不能代替本次 Kimi-K3 / 6000D / P-TP4 与 D-TP32 验证。[PR #33863](https://github.com/sgl-project/sglang/pull/33863)
+
+## 2. 为什么可以迁移
+
+DSparkDraftModel 继承 DFlashDraftModel。普通 Kimi draft 的 prompt 上下文处理都包含：采集多个 target 层的 hidden，拼接后线性投影，再归一化，生成 draft KV。
+
+#33863 将这一步改为：
+
+```text
+各 P/PP stage：采集自己负责的 hidden → 使用对应权重列做局部投影
+PP stage 间：传递并累加投影结果 [token 数, 7168]
+最后一个 P/PP stage：统一 RMSNorm → 各 draft 层 K/V 投影、K norm、RoPE → 写 draft KV
+P → D：传 target KV、KDA 状态及 draft KV
+D：使用已有 DFlash proposer / verify / commit 流程
+```
+
+依据是线性运算 `concat(h_i) × W^T = sum(h_i × W_i^T)`。归一化必须在求和后做。BF16 分段累加会改变舍入顺序，因此需要比较中间张量、logits 和 token 接受行为。
+
+关键接口已经加在共用 DFlash 模型中：[project_target_hidden_partial](https://github.com/sgl-project/sglang/blob/6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5/python/sglang/srt/models/dflash.py#L683)。最后阶段的归一化及 KV 写入实现位于 [write_projected_context_kv](https://github.com/sgl-project/sglang/blob/6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5/python/sglang/srt/models/dspark.py#L751)。
+
+这一路线保留 P 侧 PP8 的收益，不要求把各层原始 hidden 通过 PD 网络传给 D。
+
+## 3. 需要补哪些代码
+
+以下路径均相对于 SGLang 的 `python/sglang/srt/`。
+
+| 模块 | 已有基础 | 本次增量 |
+|---|---|---|
+| `models/kimi_k3.py` | PR 已支持各 PP stage 局部采集 DSpark 特征 | 接通 DFlash capture setter，按 DFlash checkpoint 的层号及 Kimi residual 语义采集 |
+| `models/dflash.py` | PR 已有局部线性投影 | 复用投影；最后阶段归一化后复用 DFlash 的逐层 KV 写入路径 |
+| `speculative/dflash_worker_v2.py` | 非 PP prefill 的 hidden → draft KV | 接收、转发 PP proxy；仅最后阶段写完整 draft KV；处理非末 stage 没有 logits/next token 的情况 |
+| `managers/scheduler_pp_mixin.py` | PR 的成功交集、失败并集及一致释放机制 | 将目前 DSpark 专用的 proxy/input 衔接扩展到 DFlash，保持请求顺序和完成条件一致 |
+| `speculative/spec_info.py` | 当前 PD 输入分派只有 EAGLE、DSpark | 接入 DFlash 输入构造，并验证 overlap FutureMap 与 idle batch 生命周期 |
+| `disaggregation/prefill.py`、`utils.py`、`mooncake/conn.py` | 最后 PP stage 传 draft KV、按层号配对 | 加入 draft GQA 的 head 分片/复制与收发端独立 stride，保留 target MLA/KDA 的已有路径 |
+| `arg_groups/speculative_hook.py`、`validation_hook.py` | 当前 main 拒绝 PP + DFlash | 仅放开已实现的 PD-prefill + PP 组合；D 端 PP1 范围不变 |
+
+关键证据：
+
+- [Kimi 局部层采集](https://github.com/sgl-project/sglang/blob/6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5/python/sglang/srt/models/kimi_k3.py#L2912)。
+- [DSpark PP 上下文处理](https://github.com/sgl-project/sglang/blob/6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5/python/sglang/srt/speculative/dspark_components/dspark_worker_v2.py#L655)。
+- [PP scheduler 的 DSpark 专用入口](https://github.com/sgl-project/sglang/blob/6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5/python/sglang/srt/managers/scheduler_pp_mixin.py#L1391)。
+- [main 的 PD 输入分派](https://github.com/sgl-project/sglang/blob/3139ceaeec50868a441d68dc231663f3777e0d93/python/sglang/srt/speculative/spec_info.py#L174)：DFlash 仍返回 None。仓库里的 `dflash_disaggregation.py` 辅助函数尚未被这个入口调用，不能仅凭文件存在认定 PD 已支持。
+
+相关 [#36140](https://github.com/sgl-project/sglang/issues/36140) 记录了输入缺失和 draft 状态缺失问题；[#36277](https://github.com/sgl-project/sglang/pull/36277) 是提前拒绝不支持配置的保护性改动，不是 DFlash PD 的实现。
+
+## 4. 异构 TP 的明确风险
+
+605 的 `/data/hf_models/Kimi-K3-DFlash/config.json` 为 6 层 draft、8 个 KV head、head_dim=128、BF16、4096 sliding window；target capture 层是 `[19,37,54,66,78,90]`。
+
+[DFlashAttention](https://github.com/sgl-project/sglang/blob/6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5/python/sglang/srt/models/dflash.py#L142) 使用 `max(1, total_kv_heads // tp_size)`：
+
+| draft 布局 | 每 rank KV heads | BF16、page=64 时单层 K 的页大小 |
+|---|---:|---:|
+| P TP4 | 2 | `64 × 2 × 128 × 2 = 32768 bytes` |
+| D TP32 | 1，按 head 复制到多个 rank | `64 × 1 × 128 × 2 = 16384 bytes` |
+
+PR 当前 [Mooncake flat 传输分支](https://github.com/sgl-project/sglang/blob/6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5/python/sglang/srt/disaggregation/mooncake/conn.py#L686) 按 layer ID 配对后，对源偏移、目标偏移和复制字节数都使用源端 `item_len`。Kimi hybrid MLA 会进入这条路径。层号正确只能确认“哪层到哪层”，没有解决“该层哪些 head 到哪个 rank”。
+
+### CPU 验证结果
+
+从该 PR 源码直接用 AST 提取 `_send_kvcache_generic` 和 `build_transfer_entry_pairs`，使用 synthetic 地址、单页索引和记录型传输函数执行。未调用 GPU、Mooncake 或 RDMA。
+
+[地址规划结果与源文件 SHA256](evidence/kimi_k3_pp_dflash_pr33863_transfer_plan.json)。
+
+| 用例 | 地址/长度是否符合目标布局 |
+|---|---|
+| 收发双方均为 1 KV head | 符合 |
+| P 为 2 head、D 为 1 head | 不符合：目标偏移和复制长度均按源端大条目计算 |
+
+具体用例：D buffer 起始地址设为 2000000，目标 page ID=2。正确目标地址为 `2000000 + 2×16384 = 2032768`，当前函数生成 `2065536`，复制 32768 bytes 而不是单 head 页的 16384 bytes。
+
+这是原 PR 的特定布局地址规划复现，未在 GPU 上故意执行越界复制。当前实现已补齐每条目布局校验及 GQA head-aware 传输，CPU 测试覆盖全部 P-TP4→D-TP32 head 映射；修复版本的完整服务已通过跨组请求和 C1/C8 评测。原 flat-copy 失败证据保留用于说明修复动机。
+
+## 5. 验收与结果口径
+
+1. **局部数学与协议验证已通过。** 覆盖各 stage capture、feature 顺序、空 capture stage、单次 RMSNorm、BF16/FP32 分段投影与 TP4→TP32 传输规划。CPU 方法级回归不等于完整模型 logits 等价验证。
+2. **最小 PD 请求已通过。** P TP4/PP8、D TP32/PP1，经 Router 返回 128 token，服务日志有实际 RDMA ready ACK，D Decode 使用 CUDA Graph。
+3. **GSM8K 固定 64 题 C1/C8 已完成。** 两轮各 64 个响应均成功，独立解析最终答案也全部通过；总耗时 195.42/55.55 秒，无 512-token 截断。完整逐题输出、直方图、服务日志和资源清理证据见结果目录。客户端非流式，不把总耗时换算成 TTFT/ITL。
+4. **统计口径已复核。** 接口的每题等权接受长度为 6.9433/7.0070；直方图按 verify 步数加权为 6.4211/6.5158。两者权重和首尾 token 处理不同，不混用。5-shot 与前 5 题重叠，排除后剩余 59 题也均通过；这仍是历史口径部署回归，不是独立无泄漏的模型能力评测。本轮不重跑 no-spec，因此不宣称投机加速倍数。
+
+真实实验代码继续维护在 601 的 `/data/hzy/sskj` 工作区，不改同事部署目录，不再创建分支。本轮核实当前检出名为 `hzy-kimi3-pd-pp8-standard`，保持现状；独立 SGLang 源码位于 `/data/hzy/src/sglang-kimi-pp-dflash-33863`。
+
+## 6. DSpark 接受长度诊断的备用路径
+
+DSpark 低接受长度的问题先独立于 PD 排查。PP + PD 适配本身不会自动修复 D-only 已存在的问题。
+
+| 检查项 | 目的 |
+|---|---|
+| checkpoint、tokenizer、mask token、target_layer_ids、RoPE | 排除 draft/target 配置失配及错误 hidden 来源 |
+| 相同 token 前缀下，逐位置 draft token 与 target logits | 找到拒绝从第几个位置开始，区分首 token 错位与持续预测质量问题 |
+| verify 的 KDA/SSM 状态更新、回滚与 token 位置 | 判断是否首次可用、随后状态漂移 |
+| fused/replay 路径与参考执行比较 | 将 kernel/状态管理错误与 drafter 本身质量分开 |
+| 接受统计定义、实际 proposal 数、bonus token | 统一接受长度口径，防止指标解释错误 |
+
+605 的 DSpark checkpoint 为 block=7、5 层、capture `[7,23,51,67,83]`；DFlash 为 block=16、6 层、另一组 capture 与 RoPE。二者的原始平均接受长度不能单独判断实现异常，需同时看每次实际提议数、逐位置接受比例和实际吞吐。
+
+本轮不重跑 no-spec 性能基线，保留此前结果作为参考。当前尚未确定 DSpark 接受长度异常的根因。

docs/evidence/kimi_k3_pp_dflash_pr33863_transfer_plan.json

@@ -0,0 +1,32 @@
+{
+  "upstream_commit": "6465a6f3d3b6c8b7fee40fba0fdc09cf5e9ca1c5",
+  "scope": "Extracted upstream functions, synthetic pointer metadata, no GPU or network I/O",
+  "source_sha256": {
+    "transfer": "2f5a73b256e040966e181a5695fb4fa97e9fc7fb5c4760832e180b0683820cea",
+    "pairing": "47bc97bfb649d6564c870113d276a1043230271f2d8924e8d41744cea12c971a"
+  },
+  "results": [
+    {
+      "case": "equal_tp_control",
+      "source_page_stride_bytes": 16384,
+      "destination_page_stride_bytes": 16384,
+      "actual_source_address": 1016384,
+      "actual_destination_address": 2032768,
+      "expected_destination_address": 2032768,
+      "actual_copy_bytes": 16384,
+      "one_destination_head_page_bytes": 16384,
+      "layout_matches": true
+    },
+    {
+      "case": "P_TP4_to_D_TP32_DFlash",
+      "source_page_stride_bytes": 32768,
+      "destination_page_stride_bytes": 16384,
+      "actual_source_address": 1032768,
+      "actual_destination_address": 2065536,
+      "expected_destination_address": 2032768,
+      "actual_copy_bytes": 32768,
+      "one_destination_head_page_bytes": 16384,
+      "layout_matches": false
+    }
+  ]
+}

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/README.md

@@ -0,0 +1,58 @@
+# Kimi-K3 DP2 无投机 A/B
+
+两个独立四节点 SGLang 副本通过 SGLang Router 做 round-robin 负载均衡。固定 `EP4`，比较：
+
+- `tp32`：每副本 `TP32/PP1/EP4`
+- `pp8`：每副本 `TP4/PP8/EP4`
+- `tp32-dflash`：每副本 `TP32/PP1/EP4`，开启 DFlash，draft tokens 16；`mem-fraction-static=0.83` 为 16K Prefill 的 FlashInfer MoE workspace 留出余量
+
+测试口径固定为 SPEED-Bench、`16K -> 512`、`C=8`、40 请求、chunk 8K、BF16 KV、FlashInfer MXFP4，并关闭 Prefix Cache。`tp32`、`pp8` 不启用投机解码；`tp32-dflash` 只增加 DFlash 及其必要参数。
+
+```bash
+cd /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab
+
+# 只展开命令
+bash run_dp2_nospec_ab.sh dry-run tp32
+bash run_dp2_nospec_ab.sh dry-run pp8
+bash run_dp2_nospec_ab.sh dry-run tp32-dflash
+
+# 完整实验
+RUN_ID=dp2-tp32-$(date +%Y%m%d-%H%M%S) bash run_dp2_nospec_ab.sh all tp32
+RUN_ID=dp2-pp8-$(date +%Y%m%d-%H%M%S) bash run_dp2_nospec_ab.sh all pp8
+RUN_ID=dp2-tp32-dflash-$(date +%Y%m%d-%H%M%S) bash run_dp2_nospec_ab.sh all tp32-dflash
+```
+
+每轮结果保存在 `results/<RUN_ID>/`。正式计时前，脚本分别给两个副本发送一个 `16K -> 1` 请求，避免 Router 只预热到一个副本。
+
+## 2026-09-02 正式结果
+
+| 模式 | Input TPS | Output TPS | TTFT mean / p95 | TPOT mean / p95 | E2E mean |
+|---|---:|---:|---:|---:|---:|
+| `2 x TP32/PP1/EP4` | 2797.64 | 85.99 | 8.35s / 20.79s | 75.93ms / 88.06ms | 47.15s |
+| `2 x TP4/PP8/EP4` | 1799.46 | 55.31 | 7.84s / 11.68s | 126.86ms / 145.13ms | 72.66s |
+| `2 x TP32/PP1/EP4 + DFlash` | 2218.94 | 68.20 | 8.12s / 21.75s | 98.44ms / 154.58ms | 58.42s |
+
+### TP32 DFlash 对照
+
+`tp32-dflash` 的投机参数与 PD+DFlash 保持一致：draft model 为 `Kimi-K3-DFlash`，draft KV 为 BF16，`speculative-num-draft-tokens=16`。Router 后的 `/server_info` 不汇总接受长度；两副本 Decode 日志共 37 个周期采样的平均接受长度为 3.76，平均接受率为 0.184。
+
+| 指标 | TP32 无投机 | TP32 + DFlash | 变化 |
+|---|---:|---:|---:|
+| Benchmark duration | 238.18s | 300.30s | +26.08% |
+| Input / Output TPS | 2797.64 / 85.99 | 2218.94 / 68.20 | -20.69% |
+| TTFT mean / p95 | 8.35s / 20.79s | 8.12s / 21.75s | -2.73% / +4.62% |
+| TPOT mean / p95 | 75.93ms / 88.06ms | 98.44ms / 154.58ms | +29.64% / +75.54% |
+| E2E mean / p95 | 47.15s / 59.25s | 58.42s / 88.76s | +23.90% / +49.83% |
+
+在当前 `16K -> 512, C=8` 口径下，DFlash 没有带来端到端收益：平均 TTFT 仅下降 2.73%，P95 TTFT 反而增加 4.62%，吞吐下降 20.69%，TPOT 和 E2E 明显恶化。当前参数不建议用于 DP2 正式部署。
+
+第一次 DFlash 正式轮使用 `mem-fraction-static=0.86`，在首个 16K Prefill 的 FlashInfer MXFP4 MoE workspace 分配阶段 OOM；DP2 单实例同时承担 Prefill 和 Decode，不能直接复用 PD 的 D-only 显存比例。仅将该模式降到 0.83 后完整通过，投机参数未改变。
+
+证据目录：
+
+- `results/dp2-tp32-final-20260902-123051/`
+- `results/dp2-pp8-final-20260902-130711/`
+- `results/dp2-tp32-dflash-mem083-20260902-143710/`
+- 首轮 OOM 证据：`results/dp2-tp32-dflash-final-20260902-141001/`
+
+三轮最终结果均为 40/40 请求成功。Router 使用固定 `round_robin`，两副本均实际处理请求。正式运行后补充了 Router 的模型目录挂载，避免后续 tokenizer 注册产生与 round-robin 转发无关的错误日志。

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/config.env

@@ -0,0 +1,36 @@
+EXPERIMENT_BASE=kimi3_pro6000_dp2_nospec_ab
+IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
+MODEL_PATH=/data/hf_models/Kimi-K3
+DRAFT_MODEL_PATH=/data/hf_models/Kimi-K3-DFlash
+DFLASH_FIX_SOURCE=/data/hzy/sskj/experiments/pro6000/cdflash_fix
+DFLASH_FIX_DIR=/data/hzy/kimi3_dp2_dflash_fix
+DFLASH_FIX_MAP=(
+  arg_groups/speculative_hook.py
+  arg_groups/validation_hook.py
+  speculative/dflash_worker_v2.py
+)
+
+REPLICA_A_NODES=(174.1.60.1 174.1.60.2 174.1.60.3 174.1.60.4)
+REPLICA_B_NODES=(174.1.60.5 174.1.60.6 174.1.60.7 174.1.60.8)
+SSH_USER=user
+SSH_OPTIONS=(-o BatchMode=yes -o ConnectTimeout=10)
+
+SERVER_PORT=30000
+DIST_PORT=20000
+ROUTER_PORT=32000
+EP_SIZE=4
+CHUNK_SIZE=8192
+PAGE_SIZE=64
+KV_DTYPE=bfloat16
+MAX_RUNNING_REQUESTS=8
+CUDA_GRAPH_MAX_BS=8
+DRAFT_TOKENS=16
+IB_DEVICES=mlx5_0,mlx5_1,mlx5_2,mlx5_3
+NCCL_DEBUG_LEVEL=${NCCL_DEBUG_LEVEL:-WARN}
+JIT_CACHE=/data/hzy/cache/kimi-dflash-fi0618-situ4460
+HEALTH_WAIT_S=2400
+
+SPEED_BENCH_DATASET=/data/speedbench16k_complete.jsonl
+NUM_PROMPTS=40
+OUTPUT_LEN=512
+CONCURRENCY=8

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/a_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_a_0 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.1 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/a_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_a_1 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.2 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/a_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_a_2 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.3 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/a_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_a_3 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.4 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/b_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_b_0 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.5 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/b_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_b_1 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.6 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/b_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_b_2 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.7 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/b_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_b_3 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.8 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/benchmark.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.1 --port 32000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 40 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 512 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 1 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/bench/pp8_speedbench16k_c8_o512.jsonl

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/router.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_pp8_router_0 --network host --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang_router.launch_router --policy round_robin --worker-urls http://174.1.60.1:30000 http://174.1.60.5:30000 --host 0.0.0.0 --port 32000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/warm_a.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.1 --port 30000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 1 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 1 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 0 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/bench/warm_a.jsonl

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/commands/warm_b.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.5 --port 30000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 1 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 1 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 0 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/bench/warm_b.jsonl

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/metadata/input_sha256.txt

@@ -0,0 +1,3 @@
+2af605e738d3a9b3f17310d54a81e81f1342b76eeeb12da7cd3f65abf6e0764f  /data/speedbench16k_complete.jsonl
+976aca142cb6183e0ed8f0fdd9395643233ce6ab9837e152f086b97c9bcfdf40  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/run_dp2_nospec_ab.sh
+28f994273a0f8b9d7ad0a3393400547c7f25cd369babc391d4a1f9553e7bcb9b  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/config.env

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/metadata/manifest.env

@@ -0,0 +1,11 @@
+RUN_ID=dp2-pp8-final-20260902-130711
+MODE=pp8
+TP_SIZE=4
+PP_SIZE=8
+EP_SIZE=4
+IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
+CHUNK_SIZE=8192
+KV_DTYPE=bfloat16
+NUM_PROMPTS=40
+OUTPUT_LEN=512
+CONCURRENCY=8

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-pp8-final-20260902-130711/metadata/preflight.txt

@@ -0,0 +1,8 @@
+174.1.60.1 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.2 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.3 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.4 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.5 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.6 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.7 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.8 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/a_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_a_0 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.1 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/a_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_a_1 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.2 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/a_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_a_2 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.3 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/a_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_a_3 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.4 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/b_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_b_0 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.5 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/b_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_b_1 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.6 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/b_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_b_2 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.7 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/b_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_b_3 --gpus all --network host --ipc=host --shm-size 32g --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -e SGLANG_HOST_IP=174.1.60.8 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e NCCL_DEBUG=WARN -e NCCL_DEBUG_SUBSYS=INIT\,ENV -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/benchmark.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.1 --port 32000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 40 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 512 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 1 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/bench/tp32_speedbench16k_c8_o512.jsonl

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/router.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_dp2_nospec_ab_tp32_router_0 --network host --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang_router.launch_router --policy round_robin --worker-urls http://174.1.60.1:30000 http://174.1.60.5:30000 --host 0.0.0.0 --port 32000

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/warm_a.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.1 --port 30000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 1 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 1 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 0 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/bench/warm_a.jsonl

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/commands/warm_b.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.5 --port 30000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 1 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 1 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 0 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/bench/warm_b.jsonl

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/metadata/input_sha256.txt

@@ -0,0 +1,3 @@
+2af605e738d3a9b3f17310d54a81e81f1342b76eeeb12da7cd3f65abf6e0764f  /data/speedbench16k_complete.jsonl
+976aca142cb6183e0ed8f0fdd9395643233ce6ab9837e152f086b97c9bcfdf40  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/run_dp2_nospec_ab.sh
+28f994273a0f8b9d7ad0a3393400547c7f25cd369babc391d4a1f9553e7bcb9b  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/config.env

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/metadata/manifest.env

@@ -0,0 +1,11 @@
+RUN_ID=dp2-tp32-final-20260902-123051
+MODE=tp32
+TP_SIZE=32
+PP_SIZE=1
+EP_SIZE=4
+IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
+CHUNK_SIZE=8192
+KV_DTYPE=bfloat16
+NUM_PROMPTS=40
+OUTPUT_LEN=512
+CONCURRENCY=8

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/results/dp2-tp32-final-20260902-123051/metadata/preflight.txt

@@ -0,0 +1,8 @@
+174.1.60.1 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.2 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.3 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.4 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.5 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.6 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.7 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.8 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f

experiments/pro6000/kimi3_pro6000_dp2_nospec_ab/run_dp2_nospec_ab.sh

@@ -0,0 +1,360 @@
+#!/usr/bin/env bash
+set -Eeuo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+source "${SCRIPT_DIR}/config.env"
+
+ACTION="${1:-status}"
+MODE="${2:-tp32}"
+DRY_RUN="${DRY_RUN:-0}"
+[[ "$MODE" == tp32 || "$MODE" == pp8 || "$MODE" == tp32-dflash ]] || {
+  echo "MODE must be tp32, pp8, or tp32-dflash" >&2
+  exit 2
+}
+
+SPEC_ENABLED=0
+if [[ "$MODE" == tp32 || "$MODE" == tp32-dflash ]]; then
+  TP_SIZE=32
+  PP_SIZE=1
+  if [[ "$MODE" == tp32-dflash ]]; then
+    MEM_FRACTION=0.83
+  else
+    MEM_FRACTION=0.86
+  fi
+  MAMBA_RATIO=0.21
+else
+  TP_SIZE=4
+  PP_SIZE=8
+  MEM_FRACTION=0.84
+  MAMBA_RATIO=0.36
+fi
+[[ "$MODE" == tp32-dflash ]] && SPEC_ENABLED=1
+
+RUN_ID="${RUN_ID:-${EXPERIMENT_BASE}-${MODE}-$(date +%Y%m%d-%H%M%S)}"
+[[ "$RUN_ID" =~ ^[a-zA-Z0-9_.-]+$ ]] || exit 2
+RESULT_ROOT="${SCRIPT_DIR}/results/${RUN_ID}"
+SUDO_PASSWORD_FILE="${SUDO_PASSWORD_FILE:-/data/hzy/.sudo_password}"
+ALL_NODES=("${REPLICA_A_NODES[@]}" "${REPLICA_B_NODES[@]}")
+mkdir -p "${RESULT_ROOT}"/{bench,commands,logs,metadata}
+
+remote() {
+  local host="$1" command
+  shift
+  if [[ "$host" == "${REPLICA_A_NODES[0]}" ]]; then
+    "$@"
+  else
+    printf -v command '%q ' "$@"
+    ssh "${SSH_OPTIONS[@]}" "${SSH_USER}@${host}" "$command"
+  fi
+}
+
+root() {
+  local host="$1" password command
+  shift
+  IFS= read -r password <"$SUDO_PASSWORD_FILE"
+  if [[ "$host" == "${REPLICA_A_NODES[0]}" ]]; then
+    printf '%s\n' "$password" | sudo -S -p '' -- "$@"
+  else
+    printf -v command '%q ' "$@"
+    printf '%s\n' "$password" | ssh "${SSH_OPTIONS[@]}" \
+      "${SSH_USER}@${host}" "sudo -S -p '' -- ${command}"
+  fi
+}
+
+container_name() {
+  printf '%s_%s_%s_%s' "$EXPERIMENT_BASE" "$MODE" "$1" "$2"
+}
+
+record_command() {
+  local key="$1"
+  shift
+  {
+    printf '%q' "$1"
+    shift
+    printf ' %q' "$@"
+    printf '\n'
+  } >"${RESULT_ROOT}/commands/${key}.cmd.txt"
+}
+
+emit() {
+  local key="$1" host="$2"
+  shift 2
+  record_command "$key" "$@"
+  printf '%s %s: ' "$key" "$host"
+  cat "${RESULT_ROOT}/commands/${key}.cmd.txt"
+  [[ "$DRY_RUN" == 1 ]] || root "$host" "$@"
+}
+
+build_server_command() {
+  local replica="$1" rank="$2" host head rel
+  local -a dflash_mounts=() dflash_args=()
+  if [[ "$replica" == a ]]; then
+    host="${REPLICA_A_NODES[$rank]}"
+    head="${REPLICA_A_NODES[0]}"
+  else
+    host="${REPLICA_B_NODES[$rank]}"
+    head="${REPLICA_B_NODES[0]}"
+  fi
+  if [[ "$SPEC_ENABLED" == 1 ]]; then
+    dflash_mounts+=(-v "${DRAFT_MODEL_PATH}:${DRAFT_MODEL_PATH}:ro")
+    for rel in "${DFLASH_FIX_MAP[@]}"; do
+      dflash_mounts+=(-v "${DFLASH_FIX_DIR}/$(basename "$rel"):/opt/kimi-dflash/python/sglang/srt/${rel}:ro")
+    done
+    dflash_args=(
+      --speculative-draft-kv-cache-dtype "$KV_DTYPE"
+      --speculative-algorithm DFLASH
+      --speculative-draft-model-path "$DRAFT_MODEL_PATH"
+      --speculative-num-draft-tokens "$DRAFT_TOKENS"
+    )
+  fi
+  HOST="$host"
+  CMD=(docker run -d --name "$(container_name "$replica" "$rank")"
+    --gpus all --network host --ipc=host --shm-size 32g
+    --ulimit memlock=-1 --device /dev/infiniband
+    -v "${MODEL_PATH}:${MODEL_PATH}:ro"
+    -v "${JIT_CACHE}:/cache"
+    "${dflash_mounts[@]}"
+    -e "SGLANG_HOST_IP=${host}" -e PYTHONUNBUFFERED=1
+    -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1
+    -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1
+    -e "NCCL_IB_HCA=${IB_DEVICES}" -e NCCL_IB_GID_INDEX=3
+    -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1
+    -e "NCCL_DEBUG=${NCCL_DEBUG_LEVEL}" -e NCCL_DEBUG_SUBSYS=INIT,ENV
+    -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
+    -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0
+    -e SGLANG_MOE_FUSED_GATE_RADIX=1
+    -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f
+    -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache
+    -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions
+    -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4
+    --entrypoint python3 "$IMAGE" -m sglang.launch_server
+    --model-path "$MODEL_PATH" --served-model-name kimi-k3 --trust-remote-code
+    --tp-size "$TP_SIZE" --pp-size "$PP_SIZE" --ep-size "$EP_SIZE"
+    --nnodes 4 --node-rank "$rank" --dist-init-addr "${head}:${DIST_PORT}"
+    --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none
+    --kv-cache-dtype "$KV_DTYPE"
+    "${dflash_args[@]}"
+    --chunked-prefill-size "$CHUNK_SIZE" --page-size "$PAGE_SIZE"
+    --mem-fraction-static "$MEM_FRACTION"
+    --mamba-full-memory-ratio "$MAMBA_RATIO"
+    --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache
+    --max-running-requests "$MAX_RUNNING_REQUESTS"
+    --cuda-graph-max-bs-decode "$CUDA_GRAPH_MAX_BS"
+    --dist-timeout 3600 --enable-metrics --enable-request-time-stats-logging
+    --host 0.0.0.0 --port "$SERVER_PORT")
+}
+
+stage_dflash_fixes() {
+  local host rel
+  [[ "$SPEC_ENABLED" == 1 ]] || return 0
+  for host in "${ALL_NODES[@]}"; do
+    remote "$host" mkdir -p "$DFLASH_FIX_DIR"
+    for rel in "${DFLASH_FIX_MAP[@]}"; do
+      if [[ "$host" == "${REPLICA_A_NODES[0]}" ]]; then
+        cp "${DFLASH_FIX_SOURCE}/$(basename "$rel")" \
+          "${DFLASH_FIX_DIR}/$(basename "$rel")"
+      else
+        scp "${SSH_OPTIONS[@]}" -q "${DFLASH_FIX_SOURCE}/$(basename "$rel")" \
+          "${SSH_USER}@${host}:${DFLASH_FIX_DIR}/$(basename "$rel")"
+      fi
+    done
+  done
+}
+
+preflight() {
+  local host image reference="" gpu ports
+  for host in "${ALL_NODES[@]}"; do
+    image="$(root "$host" docker image inspect --format '{{.Id}}' "$IMAGE")"
+    [[ -z "$reference" ]] && reference="$image"
+    [[ "$image" == "$reference" ]] || {
+      echo "image mismatch: $host" >&2
+      return 1
+    }
+    remote "$host" test -r "${MODEL_PATH}/config.json"
+    if [[ "$SPEC_ENABLED" == 1 ]]; then
+      remote "$host" test -r "${DRAFT_MODEL_PATH}/model.safetensors"
+    fi
+    remote "$host" test -e /dev/infiniband/uverbs0
+    root "$host" mkdir -p "${JIT_CACHE}/tmp"
+    gpu="$(remote "$host" nvidia-smi --query-compute-apps=pid --format=csv,noheader)"
+    [[ -z "$gpu" ]] || {
+      echo "GPU occupied: $host $gpu" >&2
+      return 1
+    }
+    ports=("$SERVER_PORT" "$DIST_PORT")
+    [[ "$host" == "${REPLICA_A_NODES[0]}" ]] && ports+=("$ROUTER_PORT")
+    remote "$host" python3 -c 'import socket,sys
+for p in map(int,sys.argv[1:]):
+ with socket.socket() as s:
+  s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
+  try:
+   s.bind(("0.0.0.0",p)); s.listen(1)
+  except OSError as exc:
+   raise SystemExit(f"TCP port {p} unavailable: {exc}")' "${ports[@]}"
+    printf '%s %s\n' "$host" "$image"
+  done >"${RESULT_ROOT}/metadata/preflight.txt"
+  test -r "$SPEED_BENCH_DATASET"
+  sha256sum "$SPEED_BENCH_DATASET" "${SCRIPT_DIR}"/*.sh \
+    "${SCRIPT_DIR}/config.env" >"${RESULT_ROOT}/metadata/input_sha256.txt"
+  {
+    printf 'RUN_ID=%s\nMODE=%s\n' "$RUN_ID" "$MODE"
+    printf 'TP_SIZE=%s\nPP_SIZE=%s\nEP_SIZE=%s\n' "$TP_SIZE" "$PP_SIZE" "$EP_SIZE"
+    printf 'IMAGE=%s\nCHUNK_SIZE=%s\nKV_DTYPE=%s\n' "$IMAGE" "$CHUNK_SIZE" "$KV_DTYPE"
+    printf 'SPEC_ENABLED=%s\nDRAFT_TOKENS=%s\n' "$SPEC_ENABLED" "$DRAFT_TOKENS"
+    printf 'NUM_PROMPTS=%s\nOUTPUT_LEN=%s\nCONCURRENCY=%s\n' \
+      "$NUM_PROMPTS" "$OUTPUT_LEN" "$CONCURRENCY"
+  } >"${RESULT_ROOT}/metadata/manifest.env"
+}
+
+wait_health() {
+  local host="$1" deadline=$((SECONDS + HEALTH_WAIT_S)) replica rank running
+  [[ "$DRY_RUN" == 1 ]] && return 0
+  while (( SECONDS < deadline )); do
+    curl -fsS --max-time 3 "http://${host}:${SERVER_PORT}/health" >/dev/null 2>&1 && return 0
+    sleep 5
+  done
+  echo "health timeout: $host:$SERVER_PORT" >&2
+  for replica in a b; do
+    for rank in 0 1 2 3; do
+      build_server_command "$replica" "$rank"
+      running="$(root "$HOST" docker inspect --format '{{.State.Running}}' \
+        "$(container_name "$replica" "$rank")" 2>/dev/null || true)"
+      printf '%s %s running=%s\n' "$replica" "$rank" "$running" >&2
+    done
+  done
+  return 1
+}
+
+start() {
+  local replica rank
+  [[ "$DRY_RUN" == 1 ]] || preflight
+  [[ "$DRY_RUN" == 1 ]] || stage_dflash_fixes
+  for replica in a b; do
+    for rank in 1 2 3 0; do
+      build_server_command "$replica" "$rank"
+      emit "${replica}_${rank}" "$HOST" "${CMD[@]}"
+    done
+    if [[ "$replica" == a ]]; then
+      wait_health "${REPLICA_A_NODES[0]}"
+    else
+      wait_health "${REPLICA_B_NODES[0]}"
+    fi
+  done
+  emit router "${REPLICA_A_NODES[0]}" docker run -d \
+    --name "$(container_name router 0)" --network host \
+    -v "${MODEL_PATH}:${MODEL_PATH}:ro" \
+    -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 \
+    --entrypoint python3 "$IMAGE" \
+    -m sglang_router.launch_router --policy round_robin \
+    --worker-urls "http://${REPLICA_A_NODES[0]}:${SERVER_PORT}" \
+      "http://${REPLICA_B_NODES[0]}:${SERVER_PORT}" \
+    --host 0.0.0.0 --port "$ROUTER_PORT"
+  [[ "$DRY_RUN" == 1 ]] || {
+    local deadline=$((SECONDS + 120))
+    until curl -fsS --max-time 3 "http://${REPLICA_A_NODES[0]}:${ROUTER_PORT}/health" >/dev/null 2>&1; do
+      (( SECONDS < deadline )) || return 1
+      sleep 2
+    done
+  }
+}
+
+bench_command() {
+  local host="$1" port="$2" prompts="$3" output_len="$4" warmups="$5" output="$6"
+  BENCH_CMD=(docker run --rm --network host -v /data:/data
+    -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1
+    --entrypoint python3 "$IMAGE" -m sglang.benchmark.serving
+    --backend sglang --host "$host" --port "$port"
+    --model "$MODEL_PATH" --tokenizer "$MODEL_PATH"
+    --dataset-name speed-bench --dataset-path "$SPEED_BENCH_DATASET"
+    --num-prompts "$prompts" --max-concurrency "$CONCURRENCY" --request-rate 10000
+    --speed-bench-output-len "$output_len"
+    --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests "$warmups"
+    --output-details --output-file "$output")
+}
+
+warm_each_replica() {
+  local replica host output log
+  for replica in a b; do
+    [[ "$replica" == a ]] && host="${REPLICA_A_NODES[0]}" || host="${REPLICA_B_NODES[0]}"
+    output="${RESULT_ROOT}/bench/warm_${replica}.jsonl"
+    log="${RESULT_ROOT}/bench/warm_${replica}.log"
+    bench_command "$host" "$SERVER_PORT" 1 1 0 "$output"
+    record_command "warm_${replica}" "${BENCH_CMD[@]}"
+    root "${REPLICA_A_NODES[0]}" "${BENCH_CMD[@]}" >"$log" 2>&1
+  done
+}
+
+benchmark() {
+  local output="${RESULT_ROOT}/bench/${MODE}_speedbench16k_c${CONCURRENCY}_o${OUTPUT_LEN}.jsonl"
+  local log="${RESULT_ROOT}/bench/${MODE}_speedbench16k_c${CONCURRENCY}_o${OUTPUT_LEN}.log"
+  warm_each_replica
+  bench_command "${REPLICA_A_NODES[0]}" "$ROUTER_PORT" \
+    "$NUM_PROMPTS" "$OUTPUT_LEN" 1 "$output"
+  record_command benchmark "${BENCH_CMD[@]}"
+  root "${REPLICA_A_NODES[0]}" "${BENCH_CMD[@]}" >"$log" 2>&1
+}
+
+logs() {
+  local replica rank
+  for replica in a b; do
+    for rank in 0 1 2 3; do
+      build_server_command "$replica" "$rank"
+      root "$HOST" docker logs "$(container_name "$replica" "$rank")" \
+        >"${RESULT_ROOT}/logs/${replica}_${rank}.log" 2>&1 || true
+    done
+  done
+  root "${REPLICA_A_NODES[0]}" docker logs "$(container_name router 0)" \
+    >"${RESULT_ROOT}/logs/router.log" 2>&1 || true
+}
+
+stop() {
+  local replica rank
+  root "${REPLICA_A_NODES[0]}" docker rm -f "$(container_name router 0)" >/dev/null 2>&1 || true
+  for replica in b a; do
+    for rank in 0 1 2 3; do
+      build_server_command "$replica" "$rank"
+      root "$HOST" docker rm -f "$(container_name "$replica" "$rank")" >/dev/null 2>&1 || true
+    done
+  done
+}
+
+on_error() {
+  local rc=$?
+  trap - ERR
+  logs || true
+  echo "FAILED rc=$rc; containers retained; evidence: $RESULT_ROOT" >&2
+  exit "$rc"
+}
+
+if [[ "$DRY_RUN" != 1 ]]; then
+  ip -o -4 addr show | grep -q " ${REPLICA_A_NODES[0]}/" || {
+    echo 'Run this entry on 601 only' >&2
+    exit 2
+  }
+  trap on_error ERR
+fi
+
+case "$ACTION" in
+  dry-run) DRY_RUN=1; start ;;
+  preflight) preflight ;;
+  start) start ;;
+  bench) benchmark; logs ;;
+  logs) logs ;;
+  stop) stop ;;
+  status)
+    for host in "${ALL_NODES[@]}"; do
+      printf '%s\n' "$host"
+      root "$host" docker ps --filter "name=${EXPERIMENT_BASE}" --format '{{.Names}} {{.Status}}'
+    done
+    ;;
+  all)
+    start
+    benchmark
+    logs
+    stop
+    ;;
+  *)
+    echo 'Usage: run_dp2_nospec_ab.sh {dry-run|preflight|start|bench|logs|stop|status|all} {tp32|pp8|tp32-dflash}' >&2
+    exit 2
+    ;;
+esac

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/config.env

@@ -0,0 +1,48 @@
+EXPERIMENT_BASE=kimi3_pro6000_pd_dflash_attribution
+PD_IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
+MODEL_PATH=/data/hf_models/Kimi-K3
+DRAFT_MODEL_PATH=/data/hf_models/Kimi-K3-DFlash
+P_NODES=(174.1.60.1 174.1.60.2 174.1.60.3 174.1.60.4)
+D_NODES=(174.1.60.5 174.1.60.6 174.1.60.7 174.1.60.8)
+SSH_USER=user
+SSH_OPTIONS=(-o BatchMode=yes -o ConnectTimeout=10)
+
+PORT=30000
+DIST_PORT=20000
+BOOTSTRAP_PORT=28800
+ROUTER_PORT=31000
+
+# The P deployment is the established PP8 TTFT optimum. D is TP32/PP1.
+P_TP=4
+P_PP=8
+P_EP=4
+D_TP=32
+D_PP=1
+D_EP=4
+P_MEM=0.84
+D_MEM=0.86
+P_MAMBA_RATIO=0.36
+D_MAMBA_RATIO=0.21
+
+CHUNK=8192
+PAGE_SIZE=64
+KV_DTYPE=bfloat16
+DRAFT_TOKENS=16
+MAX_RUNNING=8
+GRAPH_BS=8
+IB_DEVICES=mlx5_0,mlx5_1,mlx5_2,mlx5_3
+JIT_CACHE=/data/hzy/cache/kimi-dflash-fi0618-situ4460
+HEALTH_WAIT_S=2400
+
+SPEED_BENCH_DATASET=/data/speedbench16k_complete.jsonl
+BENCH_PROMPTS=40
+PROFILE_PROMPTS=8
+OUTPUT_LEN=512
+CONCURRENCY=8
+
+# Per-node temporary telemetry/profile output; copied back to the 601 result tree.
+REMOTE_WORK_BASE=/data/hzy/kimi3_pd_dflash_attribution
+TELEMETRY_DURATION_S=900
+TELEMETRY_INTERVAL_S=1
+P_PROFILE_STEPS=4
+D_PROFILE_STEPS=20

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/profile_tools.py

@@ -0,0 +1,263 @@
+#!/usr/bin/env python3
+"""Low-overhead collectors and result summarizer for Kimi-K3 PD attribution."""
+
+from __future__ import annotations
+
+import argparse
+import csv
+import json
+import re
+import statistics
+import subprocess
+import time
+from pathlib import Path
+
+
+HCA_COUNTERS = (
+    "port_xmit_data",
+    "port_rcv_data",
+    "port_xmit_wait",
+    "port_rcv_errors",
+    "port_xmit_discards",
+)
+
+
+def read_int(path: Path) -> int | None:
+    try:
+        return int(path.read_text().strip())
+    except (FileNotFoundError, PermissionError, ValueError):
+        return None
+
+
+def read_hcas() -> dict[str, dict[str, int | None]]:
+    result = {}
+    for index in range(4):
+        hca = f"mlx5_{index}"
+        root = Path(f"/sys/class/infiniband/{hca}/ports/1/counters")
+        result[hca] = {name: read_int(root / name) for name in HCA_COUNTERS}
+    return result
+
+
+def read_gpus() -> list[dict[str, float | int | str | None]]:
+    fields = (
+        "index,timestamp,utilization.gpu,memory.used,power.draw,clocks.sm,"
+        "pcie.link.gen.current,pcie.link.width.current"
+    )
+    proc = subprocess.run(
+        [
+            "nvidia-smi",
+            f"--query-gpu={fields}",
+            "--format=csv,noheader,nounits",
+        ],
+        check=False,
+        capture_output=True,
+        text=True,
+    )
+    rows = []
+    for line in proc.stdout.splitlines():
+        values = [value.strip() for value in line.split(",")]
+        if len(values) != 8:
+            continue
+        parsed: list[float | int | str | None] = [int(values[0]), values[1]]
+        for value in values[2:]:
+            try:
+                parsed.append(float(value))
+            except ValueError:
+                parsed.append(None)
+        rows.append(
+            dict(
+                zip(
+                    (
+                        "index",
+                        "timestamp",
+                        "gpu_util_pct",
+                        "memory_used_mib",
+                        "power_w",
+                        "sm_clock_mhz",
+                        "pcie_gen",
+                        "pcie_width",
+                    ),
+                    parsed,
+                )
+            )
+        )
+    return rows
+
+
+def sample(args: argparse.Namespace) -> None:
+    out = Path(args.out)
+    out.parent.mkdir(parents=True, exist_ok=True)
+    deadline = time.monotonic() + args.duration
+    with out.open("w") as handle:
+        while time.monotonic() < deadline:
+            started = time.monotonic()
+            record = {
+                "wall_time": time.time(),
+                "monotonic": started,
+                "hcas": read_hcas(),
+                "gpus": read_gpus(),
+            }
+            handle.write(json.dumps(record, separators=(",", ":")) + "\n")
+            handle.flush()
+            time.sleep(max(0.0, args.interval - (time.monotonic() - started)))
+
+
+def percentile(values: list[float], q: float) -> float | None:
+    if not values:
+        return None
+    ordered = sorted(values)
+    index = (len(ordered) - 1) * q
+    lower = int(index)
+    upper = min(lower + 1, len(ordered) - 1)
+    fraction = index - lower
+    return ordered[lower] * (1 - fraction) + ordered[upper] * fraction
+
+
+def monitor_summary(path: Path) -> dict:
+    rows = [json.loads(line) for line in path.read_text().splitlines() if line]
+    hca_rates: dict[str, dict[str, list[float]]] = {}
+    gpu_util, gpu_power = [], []
+    for previous, current in zip(rows, rows[1:]):
+        elapsed = current["monotonic"] - previous["monotonic"]
+        if elapsed <= 0:
+            continue
+        for hca, counters in current["hcas"].items():
+            target = hca_rates.setdefault(hca, {"tx_gbps": [], "rx_gbps": []})
+            old = previous["hcas"].get(hca, {})
+            for counter, output in (
+                ("port_xmit_data", "tx_gbps"),
+                ("port_rcv_data", "rx_gbps"),
+            ):
+                before, after = old.get(counter), counters.get(counter)
+                if before is not None and after is not None and after >= before:
+                    # IB port_*_data counters are in 32-bit words.
+                    target[output].append((after - before) * 4 * 8 / elapsed / 1e9)
+        for gpu in current.get("gpus", []):
+            if gpu.get("gpu_util_pct") is not None:
+                gpu_util.append(float(gpu["gpu_util_pct"]))
+            if gpu.get("power_w") is not None:
+                gpu_power.append(float(gpu["power_w"]))
+
+    hcas = {}
+    for hca, rates in hca_rates.items():
+        hcas[hca] = {
+            name: {
+                "mean": statistics.fmean(values) if values else None,
+                "p95": percentile(values, 0.95),
+                "max": max(values) if values else None,
+            }
+            for name, values in rates.items()
+        }
+    return {
+        "samples": len(rows),
+        "hcas": hcas,
+        "gpu_util_pct_mean": statistics.fmean(gpu_util) if gpu_util else None,
+        "gpu_util_pct_p95": percentile(gpu_util, 0.95),
+        "gpu_power_w_mean": statistics.fmean(gpu_power) if gpu_power else None,
+    }
+
+
+REQ_RE = re.compile(r"ReqTimeStats\(.*?type=(prefill|decode)\): (.*)$")
+VALUE_RE = re.compile(r"([a-z_]+)=([0-9.]+)(ms| GB/s| MB)?")
+
+
+def request_stats(paths: list[Path]) -> dict:
+    by_role: dict[str, dict[str, list[float]]] = {"prefill": {}, "decode": {}}
+    for path in paths:
+        for line in path.read_text(errors="replace").splitlines():
+            match = REQ_RE.search(line)
+            if not match:
+                continue
+            role, payload = match.groups()
+            for key, value, unit in VALUE_RE.findall(payload):
+                number = float(value)
+                if unit == "ms":
+                    number /= 1000
+                by_role[role].setdefault(key, []).append(number)
+    result = {}
+    for role, fields in by_role.items():
+        result[role] = {
+            key: {
+                "count": len(values),
+                "mean_s": statistics.fmean(values),
+                "p50_s": percentile(values, 0.50),
+                "p95_s": percentile(values, 0.95),
+            }
+            for key, values in fields.items()
+        }
+    return result
+
+
+def benchmark_summary(path: Path) -> dict:
+    payload = json.loads(path.read_text().splitlines()[0])
+    keys = (
+        "duration",
+        "completed",
+        "total_input_tokens",
+        "total_output_tokens",
+        "input_throughput",
+        "output_throughput",
+        "mean_ttft_ms",
+        "median_ttft_ms",
+        "p95_ttft_ms",
+        "mean_tpot_ms",
+        "median_tpot_ms",
+        "p95_tpot_ms",
+        "mean_e2e_latency_ms",
+        "p95_e2e_latency_ms",
+        "accept_length",
+    )
+    return {key: payload.get(key) for key in keys}
+
+
+def analyze(args: argparse.Namespace) -> None:
+    root = Path(args.result_root)
+    bench_files = sorted((root / "bench").glob("*.jsonl"))
+    service_logs = sorted((root / "logs").glob("[pd]_*.log"))
+    monitor_files = sorted((root / "telemetry").glob("*.jsonl"))
+    summary = {
+        "run_id": root.name,
+        "benchmarks": {path.name: benchmark_summary(path) for path in bench_files},
+        "request_stages": request_stats(service_logs),
+        "nodes": {path.stem: monitor_summary(path) for path in monitor_files},
+    }
+    (root / "summary.json").write_text(json.dumps(summary, indent=2) + "\n")
+
+    rows = []
+    for name, item in summary["benchmarks"].items():
+        rows.append(
+            {
+                "file": name,
+                "input_tps": item.get("input_throughput"),
+                "output_tps": item.get("output_throughput"),
+                "ttft_mean_ms": item.get("mean_ttft_ms"),
+                "ttft_p95_ms": item.get("p95_ttft_ms"),
+                "tpot_mean_ms": item.get("mean_tpot_ms"),
+                "e2e_mean_ms": item.get("mean_e2e_latency_ms"),
+                "accept_length": item.get("accept_length"),
+            }
+        )
+    with (root / "summary.csv").open("w", newline="") as handle:
+        writer = csv.DictWriter(handle, fieldnames=rows[0].keys() if rows else ["file"])
+        writer.writeheader()
+        writer.writerows(rows)
+    print(json.dumps(summary, indent=2))
+
+
+def main() -> None:
+    parser = argparse.ArgumentParser()
+    sub = parser.add_subparsers(dest="command", required=True)
+    collect = sub.add_parser("sample")
+    collect.add_argument("--out", required=True)
+    collect.add_argument("--interval", type=float, default=1.0)
+    collect.add_argument("--duration", type=float, default=900.0)
+    collect.set_defaults(func=sample)
+    report = sub.add_parser("analyze")
+    report.add_argument("--result-root", required=True)
+    report.set_defaults(func=analyze)
+    args = parser.parse_args()
+    args.func(args)
+
+
+if __name__ == "__main__":
+    main()

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/analysis.log

@@ -0,0 +1,550 @@
+{
+  "run_id": "kimi3-pd-dflash-ab-pmem084-20260901-1823",
+  "benchmarks": {
+    "dflash_speedbench16k_c8_o512.jsonl": {
+      "duration": 359.5340380980633,
+      "completed": 40,
+      "total_input_tokens": 666343,
+      "total_output_tokens": 20480,
+      "input_throughput": 1853.3516423784447,
+      "output_throughput": 56.96261780481006,
+      "mean_ttft_ms": 22798.979052063078,
+      "median_ttft_ms": 23065.808354876935,
+      "p95_ttft_ms": 40538.1793458946,
+      "mean_tpot_ms": 89.69703914001656,
+      "median_tpot_ms": 85.41784447767279,
+      "p95_tpot_ms": 144.75843207459573,
+      "mean_e2e_latency_ms": 68634.16605261154,
+      "p95_e2e_latency_ms": 103502.04423558897,
+      "accept_length": 3.6899555555555557
+    }
+  },
+  "request_stages": {
+    "prefill": {
+      "bootstrap_duration": {
+        "count": 368,
+        "mean_s": 5.287058940217391,
+        "p50_s": 5.410970000000001,
+        "p95_s": 9.130282499999996
+      },
+      "queue_duration": {
+        "count": 368,
+        "mean_s": 0.8010839130434783,
+        "p50_s": 0.18371500000000002,
+        "p95_s": 5.2387654999999995
+      },
+      "forward_duration": {
+        "count": 368,
+        "mean_s": 15.251485190217391,
+        "p50_s": 15.76502,
+        "p95_s": 25.584751999999998
+      },
+      "entry_time": {
+        "count": 368,
+        "mean_s": 1788259027.9312284,
+        "p50_s": 1788259065.161,
+        "p95_s": 1788259233.92095
+      },
+      "transfer_speed": {
+        "count": 368,
+        "mean_s": 0.011141304347826087,
+        "p50_s": 0.01,
+        "p95_s": 0.02
+      },
+      "transfer_total": {
+        "count": 368,
+        "mean_s": 72.08445652173914,
+        "p50_s": 66.92,
+        "p95_s": 179.11
+      }
+    },
+    "decode": {
+      "bootstrap_duration": {
+        "count": 45,
+        "mean_s": 0.007379333333333334,
+        "p50_s": 0.004719999999999999,
+        "p95_s": 0.006932
+      },
+      "alloc_wait_duration": {
+        "count": 45,
+        "mean_s": 0.004035777777777778,
+        "p50_s": 0.0016200000000000001,
+        "p95_s": 0.008291999999999992
+      },
+      "transfer_duration": {
+        "count": 45,
+        "mean_s": 20.943472,
+        "p50_s": 21.65646,
+        "p95_s": 41.664989999999996
+      },
+      "queue_duration": {
+        "count": 45,
+        "mean_s": 0.0004713333333333333,
+        "p50_s": 0.00025,
+        "p95_s": 0.001907999999999999
+      },
+      "forward_duration": {
+        "count": 45,
+        "mean_s": 41.03028822222222,
+        "p50_s": 41.494769999999995,
+        "p95_s": 73.300034
+      },
+      "entry_time": {
+        "count": 45,
+        "mean_s": 1788259062.6619778,
+        "p50_s": 1788259066.057,
+        "p95_s": 1788259232.5521998
+      }
+    }
+  },
+  "nodes": {
+    "node_1": {
+      "samples": 461,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.46754573349801914,
+            "p95": 2.661604740258578,
+            "max": 5.129448341759595
+          },
+          "rx_gbps": {
+            "mean": 0.0004972801206015601,
+            "p95": 0.0029563697009887284,
+            "max": 0.005464920348828603
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.46328415550507973,
+            "p95": 2.6362269490639667,
+            "max": 5.104022596723461
+          },
+          "rx_gbps": {
+            "mean": 0.0004917035323002276,
+            "p95": 0.0029284502572382077,
+            "max": 0.0054663911410520885
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.0305665870530996,
+            "p95": 5.581910430901064,
+            "max": 9.437913065492294
+          },
+          "rx_gbps": {
+            "mean": 0.0016606876228691106,
+            "p95": 0.009413090577393817,
+            "max": 0.014448849984259278
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.0305119052445835,
+            "p95": 5.581938367703305,
+            "max": 9.438284643229885
+          },
+          "rx_gbps": {
+            "mean": 0.0016716508735489686,
+            "p95": 0.009454284017238607,
+            "max": 0.014804783268178948
+          }
+        }
+      },
+      "gpu_util_pct_mean": 13.782065217391304,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 113.12940217391305
+    },
+    "node_2": {
+      "samples": 460,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.6875634293516228,
+            "p95": 3.7991626347437073,
+            "max": 7.534340394312826
+          },
+          "rx_gbps": {
+            "mean": 0.34688089076041023,
+            "p95": 1.9082423443874328,
+            "max": 3.8196189788520734
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.6875743803169685,
+            "p95": 3.799137912299246,
+            "max": 7.534388097774859
+          },
+          "rx_gbps": {
+            "mean": 0.34682834395317813,
+            "p95": 1.90843482194487,
+            "max": 3.8197000643054992
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.2081492413669008,
+            "p95": 6.535200415061421,
+            "max": 13.19483071256096
+          },
+          "rx_gbps": {
+            "mean": 0.002063982254672815,
+            "p95": 0.011674667156550434,
+            "max": 0.023343584341333098
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.2080991688456264,
+            "p95": 6.535403179982635,
+            "max": 13.194171736442824
+          },
+          "rx_gbps": {
+            "mean": 0.002077398078055917,
+            "p95": 0.011872061664513591,
+            "max": 0.023272913487611374
+          }
+        }
+      },
+      "gpu_util_pct_mean": 14.744553376906318,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 113.27862200435729
+    },
+    "node_3": {
+      "samples": 460,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.7111240851737107,
+            "p95": 3.857472104885116,
+            "max": 8.332141867995208
+          },
+          "rx_gbps": {
+            "mean": 0.5199013421520989,
+            "p95": 2.8621396594463224,
+            "max": 5.703847320930043
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.6695578400284031,
+            "p95": 3.6876603701251143,
+            "max": 7.823357747296769
+          },
+          "rx_gbps": {
+            "mean": 0.5198103454076894,
+            "p95": 2.8622791184801275,
+            "max": 5.703432233890121
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.3604172167616888,
+            "p95": 7.488868930655981,
+            "max": 11.254159596332075
+          },
+          "rx_gbps": {
+            "mean": 0.002485638288176322,
+            "p95": 0.014284905290714361,
+            "max": 0.022917264859717495
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.4019486090701048,
+            "p95": 7.489077219747819,
+            "max": 11.96699461911199
+          },
+          "rx_gbps": {
+            "mean": 0.002535659497956416,
+            "p95": 0.014419016139560679,
+            "max": 0.022430716580659028
+          }
+        }
+      },
+      "gpu_util_pct_mean": 15.098856209150327,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 114.67831971677559
+    },
+    "node_4": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.7130977446726517,
+            "p95": 3.8541467511340484,
+            "max": 9.184023109742789
+          },
+          "rx_gbps": {
+            "mean": 0.6944449018686529,
+            "p95": 3.815827037550314,
+            "max": 7.180653170434207
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.6714389371789723,
+            "p95": 3.6839175770278265,
+            "max": 7.822722195528127
+          },
+          "rx_gbps": {
+            "mean": 0.6943487821204123,
+            "p95": 3.815933491834194,
+            "max": 7.180778155790155
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.6689733372491813,
+            "p95": 5.015540860218739,
+            "max": 5.652665889414558
+          },
+          "rx_gbps": {
+            "mean": 0.03044467649957277,
+            "p95": 0.09281676788371253,
+            "max": 0.11268673075315219
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.6649541051471468,
+            "p95": 5.069996370153024,
+            "max": 5.826283103110275
+          },
+          "rx_gbps": {
+            "mean": 0.03044424126912499,
+            "p95": 0.09353292206253899,
+            "max": 0.11452683959556632
+          }
+        }
+      },
+      "gpu_util_pct_mean": 16.65665938864629,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 115.68646561135371
+    },
+    "node_5": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.2705242545297841,
+            "p95": 0.4287629218610861,
+            "max": 0.45776103942588925
+          },
+          "rx_gbps": {
+            "mean": 23.323180838891506,
+            "p95": 39.37542883202148,
+            "max": 42.10795696576072
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.0043960184172692224,
+            "p95": 0.016158427628872415,
+            "max": 0.021612366936421493
+          },
+          "rx_gbps": {
+            "mean": 0.7842356285075053,
+            "p95": 2.866239952440958,
+            "max": 4.957864240024164
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.54424793898122,
+            "p95": 39.22818454294181,
+            "max": 40.446478371110196
+          },
+          "rx_gbps": {
+            "mean": 1.050216688356294,
+            "p95": 3.043985180171406,
+            "max": 5.908346045175577
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.004401652902203605,
+            "p95": 0.01645620423033339,
+            "max": 0.023653747744540272
+          },
+          "rx_gbps": {
+            "mean": 0.7842340073643765,
+            "p95": 2.95875196821221,
+            "max": 5.509957824266702
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.52265283842794,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 125.78039028384279
+    },
+    "node_6": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.27066134533769753,
+            "p95": 0.4295944205935138,
+            "max": 0.4582636775716027
+          },
+          "rx_gbps": {
+            "mean": 23.32410818284097,
+            "p95": 39.37056703127179,
+            "max": 41.9164403171636
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.004351749650190087,
+            "p95": 0.01629027923135085,
+            "max": 0.025600327148794226
+          },
+          "rx_gbps": {
+            "mean": 0.7842249145636965,
+            "p95": 3.0061643968671223,
+            "max": 6.590890007715329
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.54128055413632,
+            "p95": 39.1381790552762,
+            "max": 40.41540926397
+          },
+          "rx_gbps": {
+            "mean": 1.047555740477628,
+            "p95": 3.2378173400227386,
+            "max": 6.150449572892132
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.0044017852023226665,
+            "p95": 0.016847831766977094,
+            "max": 0.024748654080415926
+          },
+          "rx_gbps": {
+            "mean": 0.7842379704309572,
+            "p95": 3.060665105874842,
+            "max": 5.756113141826065
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.26446506550218,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 124.62461790393013
+    },
+    "node_7": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.2679732612272483,
+            "p95": 0.42735075785685134,
+            "max": 0.45938530998666555
+          },
+          "rx_gbps": {
+            "mean": 23.32011762879796,
+            "p95": 39.32240769915222,
+            "max": 42.07286952901192
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.004332512948768431,
+            "p95": 0.016839465232532352,
+            "max": 0.024723539998415322
+          },
+          "rx_gbps": {
+            "mean": 0.7840873752189856,
+            "p95": 3.0087069391385306,
+            "max": 5.7842938208284185
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.542004356078866,
+            "p95": 39.22934913111186,
+            "max": 40.29561025605149
+          },
+          "rx_gbps": {
+            "mean": 1.048588462699771,
+            "p95": 3.052757228117158,
+            "max": 5.115625142036613
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.00439109639887198,
+            "p95": 0.016681764135549008,
+            "max": 0.022524431324296126
+          },
+          "rx_gbps": {
+            "mean": 0.7841433803073643,
+            "p95": 2.8527524114991984,
+            "max": 4.722818899616096
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.44705240174673,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 124.36417576419214
+    },
+    "node_8": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.26897095490867656,
+            "p95": 0.42789902958113873,
+            "max": 0.45811040306366296
+          },
+          "rx_gbps": {
+            "mean": 23.321803596647015,
+            "p95": 39.24769647313866,
+            "max": 42.37883423861629
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.0043494950195121845,
+            "p95": 0.016297844120582463,
+            "max": 0.02166967656993835
+          },
+          "rx_gbps": {
+            "mean": 0.7840019190535364,
+            "p95": 2.8610839348327373,
+            "max": 5.993408287782687
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.543425524500204,
+            "p95": 39.14487187399606,
+            "max": 40.447329742366676
+          },
+          "rx_gbps": {
+            "mean": 1.0502557192087276,
+            "p95": 3.0262610658405977,
+            "max": 5.95228011087333
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.004418839137300987,
+            "p95": 0.016579811158911825,
+            "max": 0.024433869858829522
+          },
+          "rx_gbps": {
+            "mean": 0.7839978940170476,
+            "p95": 2.7768970102114268,
+            "max": 5.577510481442915
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.30949781659389,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 122.96889737991266
+    }
+  }
+}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/bench/smoke.json

@@ -0,0 +1 @@
+{"text":" 5\nQuestion: A train travels 60 miles in 1 hour.","output_ids":[220,20,198,29328,25,401,9010,41333,220,2205,12034,306,220,16,9185,13],"meta_info":{"id":"93dcd59edfaf4c09b857ff45b677c3cf","finish_reason":{"type":"length","length":16},"prompt_tokens":22,"weight_version":"default","num_retractions":0,"forward_entry_time":1788258840.430706,"queue_time":0.0004608631134033203,"reasoning_tokens":0,"completion_tokens":16,"cached_tokens":0,"weight_versions":[{"version":"default","start":0,"end":16}],"cached_tokens_details":null,"dp_rank":null,"e2e_latency":2.939548511058092,"spec_accept_rate":0.10476190476190476,"spec_accept_length":2.2857142857142856,"spec_num_correct_drafts":11,"spec_num_proposed_drafts":105,"spec_verify_ct":7,"spec_accepted_drafts":11,"spec_proposed_drafts":105,"spec_correct_drafts_histogram":[2,2,2,0,0,1],"spec_accept_histogram":[2,2,2,0,0,1],"request_received_ts":1788258838.1645555,"api_server_dispatch_finish_ts":1788258838.166129,"request_finished_ts":1788258841.104104,"decode_throughput":3905936.064023281,"response_sent_to_client_ts":1788258841.104776}}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/benchmark.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.1 --port 31000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 40 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 512 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 1 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/bench/dflash_speedbench16k_c8_o512.jsonl 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/d_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_d_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.5 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/d_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_d_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.6 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/d_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_d_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.7 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/d_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_d_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.8 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/p_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_p_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.1 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/p_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_p_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.2 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/p_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_p_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.3 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/p_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_p_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-ab-pmem084-20260901-1823 -e SGLANG_HOST_IP=174.1.60.4 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/commands/router.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_none_router_0 --network host --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang_router.launch_router --pd-disaggregation --mini-lb --prefill http://174.1.60.1:30000 28800 --decode http://174.1.60.5:30000 --host 0.0.0.0 --port 31000 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/metadata/input_sha256.txt

@@ -0,0 +1,5 @@
+2af605e738d3a9b3f17310d54a81e81f1342b76eeeb12da7cd3f65abf6e0764f  /data/speedbench16k_complete.jsonl
+fdf4a953c93857572b6dd62eca4518e9f7d76b5c47f0397e0a915a5915e5701a  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/dflash_worker_v2_profile.py
+bc0b042e0290296da4f003b5d1a859a952d9bc0a497589bd948dc0b281fbf1b7  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/profile_tools.py
+fa1738bd56ff10718a2b83ca41297540fcd0937db51844fcdfbe3e779ae10f41  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/run_pd_dflash_attribution.sh
+5eda3098a6460d8f7450cc0334bbb9cc919ae88d02804baa2ce87cabec7850e7  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/config.env

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/metadata/manifest.env

@@ -0,0 +1,7 @@
+RUN_ID=kimi3-pd-dflash-ab-pmem084-20260901-1823
+RUN_MODE=dflash
+PROFILE_KIND=none
+IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
+CHUNK=8192
+OUTPUT_LEN=512
+CONCURRENCY=8

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/metadata/preflight.txt

@@ -0,0 +1,8 @@
+174.1.60.1 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.2 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.3 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.4 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.5 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.6 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.7 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.8 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/summary.csv

@@ -0,0 +1,2 @@
+file,input_tps,output_tps,ttft_mean_ms,ttft_p95_ms,tpot_mean_ms,e2e_mean_ms,accept_length
+dflash_speedbench16k_c8_o512.jsonl,1853.3516423784447,56.96261780481006,22798.979052063078,40538.1793458946,89.69703914001656,68634.16605261154,3.6899555555555557

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/summary.json

@@ -0,0 +1,550 @@
+{
+  "run_id": "kimi3-pd-dflash-ab-pmem084-20260901-1823",
+  "benchmarks": {
+    "dflash_speedbench16k_c8_o512.jsonl": {
+      "duration": 359.5340380980633,
+      "completed": 40,
+      "total_input_tokens": 666343,
+      "total_output_tokens": 20480,
+      "input_throughput": 1853.3516423784447,
+      "output_throughput": 56.96261780481006,
+      "mean_ttft_ms": 22798.979052063078,
+      "median_ttft_ms": 23065.808354876935,
+      "p95_ttft_ms": 40538.1793458946,
+      "mean_tpot_ms": 89.69703914001656,
+      "median_tpot_ms": 85.41784447767279,
+      "p95_tpot_ms": 144.75843207459573,
+      "mean_e2e_latency_ms": 68634.16605261154,
+      "p95_e2e_latency_ms": 103502.04423558897,
+      "accept_length": 3.6899555555555557
+    }
+  },
+  "request_stages": {
+    "prefill": {
+      "bootstrap_duration": {
+        "count": 368,
+        "mean_s": 5.287058940217391,
+        "p50_s": 5.410970000000001,
+        "p95_s": 9.130282499999996
+      },
+      "queue_duration": {
+        "count": 368,
+        "mean_s": 0.8010839130434783,
+        "p50_s": 0.18371500000000002,
+        "p95_s": 5.2387654999999995
+      },
+      "forward_duration": {
+        "count": 368,
+        "mean_s": 15.251485190217391,
+        "p50_s": 15.76502,
+        "p95_s": 25.584751999999998
+      },
+      "entry_time": {
+        "count": 368,
+        "mean_s": 1788259027.9312284,
+        "p50_s": 1788259065.161,
+        "p95_s": 1788259233.92095
+      },
+      "transfer_speed": {
+        "count": 368,
+        "mean_s": 0.011141304347826087,
+        "p50_s": 0.01,
+        "p95_s": 0.02
+      },
+      "transfer_total": {
+        "count": 368,
+        "mean_s": 72.08445652173914,
+        "p50_s": 66.92,
+        "p95_s": 179.11
+      }
+    },
+    "decode": {
+      "bootstrap_duration": {
+        "count": 45,
+        "mean_s": 0.007379333333333334,
+        "p50_s": 0.004719999999999999,
+        "p95_s": 0.006932
+      },
+      "alloc_wait_duration": {
+        "count": 45,
+        "mean_s": 0.004035777777777778,
+        "p50_s": 0.0016200000000000001,
+        "p95_s": 0.008291999999999992
+      },
+      "transfer_duration": {
+        "count": 45,
+        "mean_s": 20.943472,
+        "p50_s": 21.65646,
+        "p95_s": 41.664989999999996
+      },
+      "queue_duration": {
+        "count": 45,
+        "mean_s": 0.0004713333333333333,
+        "p50_s": 0.00025,
+        "p95_s": 0.001907999999999999
+      },
+      "forward_duration": {
+        "count": 45,
+        "mean_s": 41.03028822222222,
+        "p50_s": 41.494769999999995,
+        "p95_s": 73.300034
+      },
+      "entry_time": {
+        "count": 45,
+        "mean_s": 1788259062.6619778,
+        "p50_s": 1788259066.057,
+        "p95_s": 1788259232.5521998
+      }
+    }
+  },
+  "nodes": {
+    "node_1": {
+      "samples": 461,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.46754573349801914,
+            "p95": 2.661604740258578,
+            "max": 5.129448341759595
+          },
+          "rx_gbps": {
+            "mean": 0.0004972801206015601,
+            "p95": 0.0029563697009887284,
+            "max": 0.005464920348828603
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.46328415550507973,
+            "p95": 2.6362269490639667,
+            "max": 5.104022596723461
+          },
+          "rx_gbps": {
+            "mean": 0.0004917035323002276,
+            "p95": 0.0029284502572382077,
+            "max": 0.0054663911410520885
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.0305665870530996,
+            "p95": 5.581910430901064,
+            "max": 9.437913065492294
+          },
+          "rx_gbps": {
+            "mean": 0.0016606876228691106,
+            "p95": 0.009413090577393817,
+            "max": 0.014448849984259278
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.0305119052445835,
+            "p95": 5.581938367703305,
+            "max": 9.438284643229885
+          },
+          "rx_gbps": {
+            "mean": 0.0016716508735489686,
+            "p95": 0.009454284017238607,
+            "max": 0.014804783268178948
+          }
+        }
+      },
+      "gpu_util_pct_mean": 13.782065217391304,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 113.12940217391305
+    },
+    "node_2": {
+      "samples": 460,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.6875634293516228,
+            "p95": 3.7991626347437073,
+            "max": 7.534340394312826
+          },
+          "rx_gbps": {
+            "mean": 0.34688089076041023,
+            "p95": 1.9082423443874328,
+            "max": 3.8196189788520734
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.6875743803169685,
+            "p95": 3.799137912299246,
+            "max": 7.534388097774859
+          },
+          "rx_gbps": {
+            "mean": 0.34682834395317813,
+            "p95": 1.90843482194487,
+            "max": 3.8197000643054992
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.2081492413669008,
+            "p95": 6.535200415061421,
+            "max": 13.19483071256096
+          },
+          "rx_gbps": {
+            "mean": 0.002063982254672815,
+            "p95": 0.011674667156550434,
+            "max": 0.023343584341333098
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.2080991688456264,
+            "p95": 6.535403179982635,
+            "max": 13.194171736442824
+          },
+          "rx_gbps": {
+            "mean": 0.002077398078055917,
+            "p95": 0.011872061664513591,
+            "max": 0.023272913487611374
+          }
+        }
+      },
+      "gpu_util_pct_mean": 14.744553376906318,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 113.27862200435729
+    },
+    "node_3": {
+      "samples": 460,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.7111240851737107,
+            "p95": 3.857472104885116,
+            "max": 8.332141867995208
+          },
+          "rx_gbps": {
+            "mean": 0.5199013421520989,
+            "p95": 2.8621396594463224,
+            "max": 5.703847320930043
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.6695578400284031,
+            "p95": 3.6876603701251143,
+            "max": 7.823357747296769
+          },
+          "rx_gbps": {
+            "mean": 0.5198103454076894,
+            "p95": 2.8622791184801275,
+            "max": 5.703432233890121
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.3604172167616888,
+            "p95": 7.488868930655981,
+            "max": 11.254159596332075
+          },
+          "rx_gbps": {
+            "mean": 0.002485638288176322,
+            "p95": 0.014284905290714361,
+            "max": 0.022917264859717495
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.4019486090701048,
+            "p95": 7.489077219747819,
+            "max": 11.96699461911199
+          },
+          "rx_gbps": {
+            "mean": 0.002535659497956416,
+            "p95": 0.014419016139560679,
+            "max": 0.022430716580659028
+          }
+        }
+      },
+      "gpu_util_pct_mean": 15.098856209150327,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 114.67831971677559
+    },
+    "node_4": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.7130977446726517,
+            "p95": 3.8541467511340484,
+            "max": 9.184023109742789
+          },
+          "rx_gbps": {
+            "mean": 0.6944449018686529,
+            "p95": 3.815827037550314,
+            "max": 7.180653170434207
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.6714389371789723,
+            "p95": 3.6839175770278265,
+            "max": 7.822722195528127
+          },
+          "rx_gbps": {
+            "mean": 0.6943487821204123,
+            "p95": 3.815933491834194,
+            "max": 7.180778155790155
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 1.6689733372491813,
+            "p95": 5.015540860218739,
+            "max": 5.652665889414558
+          },
+          "rx_gbps": {
+            "mean": 0.03044467649957277,
+            "p95": 0.09281676788371253,
+            "max": 0.11268673075315219
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 1.6649541051471468,
+            "p95": 5.069996370153024,
+            "max": 5.826283103110275
+          },
+          "rx_gbps": {
+            "mean": 0.03044424126912499,
+            "p95": 0.09353292206253899,
+            "max": 0.11452683959556632
+          }
+        }
+      },
+      "gpu_util_pct_mean": 16.65665938864629,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 115.68646561135371
+    },
+    "node_5": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.2705242545297841,
+            "p95": 0.4287629218610861,
+            "max": 0.45776103942588925
+          },
+          "rx_gbps": {
+            "mean": 23.323180838891506,
+            "p95": 39.37542883202148,
+            "max": 42.10795696576072
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.0043960184172692224,
+            "p95": 0.016158427628872415,
+            "max": 0.021612366936421493
+          },
+          "rx_gbps": {
+            "mean": 0.7842356285075053,
+            "p95": 2.866239952440958,
+            "max": 4.957864240024164
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.54424793898122,
+            "p95": 39.22818454294181,
+            "max": 40.446478371110196
+          },
+          "rx_gbps": {
+            "mean": 1.050216688356294,
+            "p95": 3.043985180171406,
+            "max": 5.908346045175577
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.004401652902203605,
+            "p95": 0.01645620423033339,
+            "max": 0.023653747744540272
+          },
+          "rx_gbps": {
+            "mean": 0.7842340073643765,
+            "p95": 2.95875196821221,
+            "max": 5.509957824266702
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.52265283842794,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 125.78039028384279
+    },
+    "node_6": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.27066134533769753,
+            "p95": 0.4295944205935138,
+            "max": 0.4582636775716027
+          },
+          "rx_gbps": {
+            "mean": 23.32410818284097,
+            "p95": 39.37056703127179,
+            "max": 41.9164403171636
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.004351749650190087,
+            "p95": 0.01629027923135085,
+            "max": 0.025600327148794226
+          },
+          "rx_gbps": {
+            "mean": 0.7842249145636965,
+            "p95": 3.0061643968671223,
+            "max": 6.590890007715329
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.54128055413632,
+            "p95": 39.1381790552762,
+            "max": 40.41540926397
+          },
+          "rx_gbps": {
+            "mean": 1.047555740477628,
+            "p95": 3.2378173400227386,
+            "max": 6.150449572892132
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.0044017852023226665,
+            "p95": 0.016847831766977094,
+            "max": 0.024748654080415926
+          },
+          "rx_gbps": {
+            "mean": 0.7842379704309572,
+            "p95": 3.060665105874842,
+            "max": 5.756113141826065
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.26446506550218,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 124.62461790393013
+    },
+    "node_7": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.2679732612272483,
+            "p95": 0.42735075785685134,
+            "max": 0.45938530998666555
+          },
+          "rx_gbps": {
+            "mean": 23.32011762879796,
+            "p95": 39.32240769915222,
+            "max": 42.07286952901192
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.004332512948768431,
+            "p95": 0.016839465232532352,
+            "max": 0.024723539998415322
+          },
+          "rx_gbps": {
+            "mean": 0.7840873752189856,
+            "p95": 3.0087069391385306,
+            "max": 5.7842938208284185
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.542004356078866,
+            "p95": 39.22934913111186,
+            "max": 40.29561025605149
+          },
+          "rx_gbps": {
+            "mean": 1.048588462699771,
+            "p95": 3.052757228117158,
+            "max": 5.115625142036613
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.00439109639887198,
+            "p95": 0.016681764135549008,
+            "max": 0.022524431324296126
+          },
+          "rx_gbps": {
+            "mean": 0.7841433803073643,
+            "p95": 2.8527524114991984,
+            "max": 4.722818899616096
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.44705240174673,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 124.36417576419214
+    },
+    "node_8": {
+      "samples": 459,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.26897095490867656,
+            "p95": 0.42789902958113873,
+            "max": 0.45811040306366296
+          },
+          "rx_gbps": {
+            "mean": 23.321803596647015,
+            "p95": 39.24769647313866,
+            "max": 42.37883423861629
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.0043494950195121845,
+            "p95": 0.016297844120582463,
+            "max": 0.02166967656993835
+          },
+          "rx_gbps": {
+            "mean": 0.7840019190535364,
+            "p95": 2.8610839348327373,
+            "max": 5.993408287782687
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 22.543425524500204,
+            "p95": 39.14487187399606,
+            "max": 40.447329742366676
+          },
+          "rx_gbps": {
+            "mean": 1.0502557192087276,
+            "p95": 3.0262610658405977,
+            "max": 5.95228011087333
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.004418839137300987,
+            "p95": 0.016579811158911825,
+            "max": 0.024433869858829522
+          },
+          "rx_gbps": {
+            "mean": 0.7839978940170476,
+            "p95": 2.7768970102114268,
+            "max": 5.577510481442915
+          }
+        }
+      },
+      "gpu_util_pct_mean": 74.30949781659389,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 122.96889737991266
+    }
+  }
+}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_1.jsonl

@@ -0,0 +1,461 @@
+{"wall_time":1788258841.1714463,"monotonic":2349716.16777636,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:01.332","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:01.344","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.28,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:01.355","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:01.366","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:01.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:01.387","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:01.398","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:01.409","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258842.1717076,"monotonic":2349717.168037202,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:02.321","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:02.332","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:02.343","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:02.354","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:02.364","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:02.373","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:02.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:02.392","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258843.172483,"monotonic":2349718.168812387,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:03.336","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:03.348","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:03.359","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:03.370","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:03.380","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:03.389","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:03.399","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:03.409","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258844.1732361,"monotonic":2349719.169565473,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:04.320","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:04.332","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:04.343","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:04.354","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:04.364","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:04.375","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:04.386","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:04.396","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258845.1740043,"monotonic":2349720.170333857,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.308","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.320","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.331","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.343","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.355","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.366","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.377","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.387","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.1747944,"monotonic":2349721.171122112,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.321","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.333","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.344","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.355","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.365","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.375","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.384","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.394","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.175588,"monotonic":2349722.171915033,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.338","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.349","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.360","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.371","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.395","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.406","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.417","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.1757095,"monotonic":2349723.172038227,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.321","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.332","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.343","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.354","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.364","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.373","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.382","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.391","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.1765018,"monotonic":2349724.172830106,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.337","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.349","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.359","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.370","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.380","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.390","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.399","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.408","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.1772635,"monotonic":2349725.173592391,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.316","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.328","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.339","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.350","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.360","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.369","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.379","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.388","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.1780543,"monotonic":2349726.174382812,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.330","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.341","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.19,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.353","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.364","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.375","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.388","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.402","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.415","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.1788034,"monotonic":2349727.175131783,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.296","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.307","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.318","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.329","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.338","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.348","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.357","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.366","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.1796076,"monotonic":2349728.175936946,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.323","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.334","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.346","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.358","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.371","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.395","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.406","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.1803691,"monotonic":2349729.176697971,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.323","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.335","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.346","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.356","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.367","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.385","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.394","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.1811466,"monotonic":2349730.17747628,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.334","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.345","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.356","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.367","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.377","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.387","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.398","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.409","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.1819117,"monotonic":2349731.178240785,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.327","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.339","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.350","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.360","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.370","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.379","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.389","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.397","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.1826935,"monotonic":2349732.179021795,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.327","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.338","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.349","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.360","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.371","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.382","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.393","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.404","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":82.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.1834612,"monotonic":2349733.179789687,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.356","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.368","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.379","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.389","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.399","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.408","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.417","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.426","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.184211,"monotonic":2349734.180539866,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.339","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.352","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.364","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.376","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.388","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.404","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.421","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.437","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.184938,"monotonic":2349735.181267415,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.303","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.315","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.331","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.352","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.399","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.419","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.431","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.1856732,"monotonic":2349736.182002697,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.332","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.344","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.355","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.366","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.385","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.396","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.406","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.1859853,"monotonic":2349737.182312088,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.328","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.339","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.350","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.361","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.371","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.380","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.389","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.398","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.1867821,"monotonic":2349738.183109435,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.334","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.346","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.357","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.368","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.380","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.392","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.403","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.414","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.1875398,"monotonic":2349739.183868113,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.331","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.343","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.354","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.364","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.373","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.392","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.401","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.1883225,"monotonic":2349740.184650984,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.317","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.329","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.340","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.351","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.363","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.375","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.386","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.397","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.1891015,"monotonic":2349741.185430413,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.311","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.322","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.333","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.343","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.353","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.362","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.372","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.381","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":82.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.1899312,"monotonic":2349742.186258229,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.340","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.352","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.363","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.375","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.387","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.399","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.410","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.422","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.190684,"monotonic":2349743.187012525,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.328","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.339","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.350","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.361","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.372","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.381","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.391","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.400","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.191471,"monotonic":2349744.187799922,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.338","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.350","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.361","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.372","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.384","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.395","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.407","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.421","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.1922333,"monotonic":2349745.188561571,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.273","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.285","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.295","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.305","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.315","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.324","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.333","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.342","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.193083,"monotonic":2349746.189411868,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.332","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.343","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.354","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.365","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.377","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.388","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.400","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.410","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.193871,"monotonic":2349747.190198508,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.341","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.352","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.363","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.373","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.382","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.392","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.401","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.410","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.194651,"monotonic":2349748.190979628,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.313","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.322","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.330","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.338","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.347","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.355","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.364","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.371","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.1954677,"monotonic":2349749.191797056,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.309","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.317","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.325","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.333","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.340","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.347","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.354","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.360","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.195706,"monotonic":2349750.192034939,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.299","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.308","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.316","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.323","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.331","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.338","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.346","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.354","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.1965544,"monotonic":2349751.192882611,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.329","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.338","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.346","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.354","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.362","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.369","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.1973839,"monotonic":2349752.193711816,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.313","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.321","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.330","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.339","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.349","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.358","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.367","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:37.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.1982005,"monotonic":2349753.194529772,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.333","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.341","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.349","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.357","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.365","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.372","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.379","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.386","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.1990142,"monotonic":2349754.195342777,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.313","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.321","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.331","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.340","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.349","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.358","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.366","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.1998222,"monotonic":2349755.196151188,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.313","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.322","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.330","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.337","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.345","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.352","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.359","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.365","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.2001958,"monotonic":2349756.196523495,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.314","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.324","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.333","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.342","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.354","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.367","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.380","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:41.391","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.200991,"monotonic":2349757.197319789,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.333","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.342","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.350","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.358","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.366","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.373","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.380","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.386","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":82.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.2018063,"monotonic":2349758.198135693,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.314","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.324","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.336","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.348","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.361","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.373","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.381","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.388","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.2025952,"monotonic":2349759.198924802,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.337","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.345","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.353","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.361","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.369","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.389","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.2033863,"monotonic":2349760.19971573,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.340","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.354","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.366","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.374","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.382","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.389","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.396","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.403","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.2041805,"monotonic":2349761.200510031,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.337","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":78.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.346","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.353","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.361","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.369","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.376","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.389","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.205021,"monotonic":2349762.201347656,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.345","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":103.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.356","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":102.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.367","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":93.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.378","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":92.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.387","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.398","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.408","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:47.419","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.2058163,"monotonic":2349763.202143691,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.307","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":157.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.318","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":105.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.329","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":103.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.340","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":98.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.350","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.360","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.369","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.378","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.2066386,"monotonic":2349764.20296741,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.354","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":82.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.365","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.377","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.390","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.402","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.414","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.426","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.439","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.2073946,"monotonic":2349765.203723761,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.328","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.341","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.352","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.362","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.372","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.381","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.390","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.399","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.2082007,"monotonic":2349766.204529168,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.363","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.376","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.388","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.400","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.412","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.423","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.434","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:51.450","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.2089293,"monotonic":2349767.205257871,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.290","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.301","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.312","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.322","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.332","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.342","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.351","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.360","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.2097838,"monotonic":2349768.206112093,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.366","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.380","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.392","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.403","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.415","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.427","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.439","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.450","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.210533,"monotonic":2349769.206861944,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.310","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.323","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.334","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.345","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.355","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.365","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.374","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.383","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.2113516,"monotonic":2349770.20768107,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.380","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.392","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.405","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.417","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.428","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.444","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.461","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.478","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.2120655,"monotonic":2349771.208395303,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.310","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.322","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.333","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.344","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.354","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.364","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.373","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.382","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.2129111,"monotonic":2349772.209238635,"hcas":{"mlx5_0":{"port_xmit_data":138731182951814,"port_rcv_data":138763965579955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830451276935,"port_rcv_data":1192111360763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842080601158,"port_rcv_data":139149735611531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172117066245,"port_rcv_data":1505234592457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.367","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":118.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.394","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":175.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.409","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":167.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.424","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":124.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.438","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.451","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.462","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.474","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.2136414,"monotonic":2349773.209969592,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842169663673,"port_rcv_data":139149735786649,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172214998230,"port_rcv_data":1505234772049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.310","gpu_util_pct":86.0,"memory_used_mib":59345.0,"power_w":324.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.320","gpu_util_pct":31.0,"memory_used_mib":59345.0,"power_w":303.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.329","gpu_util_pct":94.0,"memory_used_mib":59345.0,"power_w":301.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.337","gpu_util_pct":68.0,"memory_used_mib":59345.0,"power_w":304.79,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.345","gpu_util_pct":99.0,"memory_used_mib":64333.0,"power_w":153.89,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.353","gpu_util_pct":100.0,"memory_used_mib":64333.0,"power_w":163.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.361","gpu_util_pct":37.0,"memory_used_mib":64333.0,"power_w":171.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.369","gpu_util_pct":100.0,"memory_used_mib":64333.0,"power_w":168.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.2144833,"monotonic":2349774.210812401,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.339","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":152.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.352","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":91.96,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.364","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":82.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.376","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":105.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.388","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":238.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.400","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":258.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.416","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":272.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.436","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":255.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.2152262,"monotonic":2349775.211555452,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.341","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.353","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.57,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.364","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.375","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.385","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":99.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.395","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":109.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.404","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.413","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":109.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.2156994,"monotonic":2349776.212028476,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.340","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":132.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.355","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.372","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.389","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.405","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.416","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.427","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.438","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.2164705,"monotonic":2349777.212799241,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.339","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.351","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.362","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.372","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.383","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.392","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.402","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.411","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.2172685,"monotonic":2349778.213597675,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.346","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.358","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.369","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.381","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.393","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.405","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.416","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.427","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.2180448,"monotonic":2349779.214373486,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.298","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.309","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.320","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.331","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.341","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.350","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.359","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.368","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.2188883,"monotonic":2349780.215217049,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.323","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.335","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.351","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.367","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.384","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.397","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.408","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.418","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.219664,"monotonic":2349781.215992666,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.323","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.335","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.346","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.356","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.366","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.376","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.385","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.394","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.2204843,"monotonic":2349782.216813279,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.326","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.338","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.350","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.362","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.374","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.391","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.407","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.425","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.2212439,"monotonic":2349783.217573147,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842255120573,"port_rcv_data":139149735868195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172291583190,"port_rcv_data":1505234846620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.343","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.355","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.366","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.377","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.386","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.396","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.406","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.415","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.2220566,"monotonic":2349784.218385373,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.349","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":132.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.361","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.373","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.386","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.397","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.409","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.425","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.442","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":83.16,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.222795,"monotonic":2349785.219123956,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.303","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.314","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.325","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.336","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.348","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.359","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.368","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.378","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.223627,"monotonic":2349786.219956638,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.355","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.368","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.381","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.393","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.405","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.417","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:11.429","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:11.444","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.224361,"monotonic":2349787.220690706,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.326","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.338","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.349","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.359","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.369","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.379","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.390","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.400","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.2251532,"monotonic":2349788.221483095,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.327","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.339","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.352","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.364","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.375","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.391","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.409","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.426","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.2259278,"monotonic":2349789.222256972,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.306","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":132.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.318","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.329","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.340","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.350","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.360","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.369","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.378","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.226773,"monotonic":2349790.223101778,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.348","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":131.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.361","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.378","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":81.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.394","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.411","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.426","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.434","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:15.442","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.2275264,"monotonic":2349791.223856087,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.331","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":105.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.343","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.354","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.365","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.375","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.385","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.394","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.403","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.22833,"monotonic":2349792.224658987,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.353","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.367","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.379","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.395","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.411","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.427","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.443","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.454","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.2290916,"monotonic":2349793.225420298,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.330","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.342","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.353","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.363","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.373","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.382","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.391","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.400","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.2299151,"monotonic":2349794.226244475,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.356","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.372","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.388","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.405","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.418","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:19.429","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:19.440","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:19.450","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.2306874,"monotonic":2349795.227016441,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.335","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.347","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.358","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.368","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.378","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.387","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.396","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.405","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.2315078,"monotonic":2349796.227836779,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.334","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.346","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.358","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.370","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.381","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.396","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:21.413","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:21.430","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.2316968,"monotonic":2349797.228025987,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.335","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.347","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.359","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.371","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.383","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:22.398","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.411","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.422","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.232496,"monotonic":2349798.228825488,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.331","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.344","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.358","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.371","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.379","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.388","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.396","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.404","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.233315,"monotonic":2349799.22964157,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.335","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.347","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.358","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.369","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.379","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.388","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:24.398","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:24.407","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.234146,"monotonic":2349800.230474174,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.385","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.402","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.413","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.423","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.433","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.443","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.452","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.461","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.2348895,"monotonic":2349801.231219018,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.335","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.346","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.357","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.368","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.377","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.387","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.396","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.405","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.2357125,"monotonic":2349802.232041954,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.318","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.330","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.342","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.354","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.366","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.382","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.398","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:27.415","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.2364943,"monotonic":2349803.232823838,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.338","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.350","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.361","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.371","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.381","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.390","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.399","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.408","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.2373056,"monotonic":2349804.233634732,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.354","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.366","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.378","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.390","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.402","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.414","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:29.426","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:29.437","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.2380898,"monotonic":2349805.234418346,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.318","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.329","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.340","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.351","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.361","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.370","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.380","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.388","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.2389476,"monotonic":2349806.235275562,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.342","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.353","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.365","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.378","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.390","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.402","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.414","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.425","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.2397473,"monotonic":2349807.236076331,"hcas":{"mlx5_0":{"port_xmit_data":138731259536774,"port_rcv_data":138763965655487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.339","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.351","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.362","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.372","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.382","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.391","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.400","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.409","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.2405457,"monotonic":2349808.236872705,"hcas":{"mlx5_0":{"port_xmit_data":138731259536974,"port_rcv_data":138763965655635,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830527861895,"port_rcv_data":1192111436263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842311507517,"port_rcv_data":139149736013169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172347943921,"port_rcv_data":1505234994670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.365","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.378","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.396","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.414","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.431","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.448","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.460","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.476","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.2413177,"monotonic":2349809.237645336,"hcas":{"mlx5_0":{"port_xmit_data":138731339793407,"port_rcv_data":138763965745648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830607324455,"port_rcv_data":1192111523838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842426737388,"port_rcv_data":139149736135671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172463171741,"port_rcv_data":1505235116369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.378","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.390","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.47,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.401","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.412","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.421","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.431","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.37,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:34.440","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:34.449","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.2421093,"monotonic":2349810.238436879,"hcas":{"mlx5_0":{"port_xmit_data":138731339793407,"port_rcv_data":138763965745648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830607324455,"port_rcv_data":1192111523838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842426737388,"port_rcv_data":139149736135671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172463171741,"port_rcv_data":1505235116369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.355","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.367","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.52,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.378","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.390","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.402","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.414","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.426","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.438","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.2429028,"monotonic":2349811.239231478,"hcas":{"mlx5_0":{"port_xmit_data":138731339793407,"port_rcv_data":138763965745648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830607324455,"port_rcv_data":1192111523838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842426737388,"port_rcv_data":139149736135671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172463171741,"port_rcv_data":1505235116369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.387","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.399","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.410","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.421","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.431","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.441","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:36.451","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:36.460","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.2433338,"monotonic":2349812.23966183,"hcas":{"mlx5_0":{"port_xmit_data":138731339793407,"port_rcv_data":138763965745648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830607324455,"port_rcv_data":1192111523838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842426737388,"port_rcv_data":139149736135671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172463171741,"port_rcv_data":1505235116369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.353","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.365","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.376","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.386","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.396","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.407","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.418","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.429","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.2437136,"monotonic":2349813.240042083,"hcas":{"mlx5_0":{"port_xmit_data":138731339793407,"port_rcv_data":138763965745648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830607324455,"port_rcv_data":1192111523838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842426737388,"port_rcv_data":139149736135671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172463171741,"port_rcv_data":1505235116369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.393","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.418","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.55,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.439","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.465","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.478","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.490","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.501","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.512","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.2444358,"monotonic":2349814.240765291,"hcas":{"mlx5_0":{"port_xmit_data":138731339793407,"port_rcv_data":138763965745648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830607324455,"port_rcv_data":1192111523838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842426737388,"port_rcv_data":139149736135671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172463171741,"port_rcv_data":1505235116369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.387","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.400","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.412","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.424","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.436","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.448","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.459","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.475","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.2451966,"monotonic":2349815.241524649,"hcas":{"mlx5_0":{"port_xmit_data":138731339793407,"port_rcv_data":138763965745648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830607324455,"port_rcv_data":1192111523838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842426737388,"port_rcv_data":139149736135671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172463171741,"port_rcv_data":1505235116369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.347","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":158.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.360","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":156.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.372","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":149.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.383","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":150.51,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.395","gpu_util_pct":0.0,"memory_used_mib":64445.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.406","gpu_util_pct":0.0,"memory_used_mib":64445.0,"power_w":78.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:40.416","gpu_util_pct":0.0,"memory_used_mib":64445.0,"power_w":78.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.426","gpu_util_pct":0.0,"memory_used_mib":64445.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.2460036,"monotonic":2349816.242331976,"hcas":{"mlx5_0":{"port_xmit_data":138731416378367,"port_rcv_data":138763965820947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830683909415,"port_rcv_data":1192111599416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842601278253,"port_rcv_data":139149736412839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172637749429,"port_rcv_data":1505235430515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.374","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":337.45,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.389","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":319.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.402","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":336.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.415","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":356.34,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.428","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":200.66,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:41.446","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":194.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.462","gpu_util_pct":99.0,"memory_used_mib":64445.0,"power_w":308.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.483","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":321.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.2467523,"monotonic":2349817.243080293,"hcas":{"mlx5_0":{"port_xmit_data":138731492963327,"port_rcv_data":138763965895951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830760494375,"port_rcv_data":1192111674452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842657663760,"port_rcv_data":139149736558981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172694116765,"port_rcv_data":1505235587402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.350","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":283.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.363","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":267.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.374","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":290.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.387","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":279.21,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:42.397","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":345.41,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.407","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":346.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.416","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":320.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.425","gpu_util_pct":87.0,"memory_used_mib":64445.0,"power_w":361.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.2475772,"monotonic":2349818.24390504,"hcas":{"mlx5_0":{"port_xmit_data":138731569548287,"port_rcv_data":138763965971080,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830837079335,"port_rcv_data":1192111749704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138842832165657,"port_rcv_data":139149736798623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2172868625130,"port_rcv_data":1505235832922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.349","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":235.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.362","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":232.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.375","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":206.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.389","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":232.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.402","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":280.37,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.415","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":302.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.429","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":285.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.442","gpu_util_pct":95.0,"memory_used_mib":64557.0,"power_w":300.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.248361,"monotonic":2349819.24468974,"hcas":{"mlx5_0":{"port_xmit_data":138731646133247,"port_rcv_data":138763966046472,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830913664295,"port_rcv_data":1192111825205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138843036039695,"port_rcv_data":139149737171510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2173072473214,"port_rcv_data":1505236195544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.393","gpu_util_pct":94.0,"memory_used_mib":59345.0,"power_w":281.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.406","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":257.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.421","gpu_util_pct":98.0,"memory_used_mib":59345.0,"power_w":267.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.436","gpu_util_pct":95.0,"memory_used_mib":59345.0,"power_w":266.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.449","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":257.28,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.463","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":243.11,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.476","gpu_util_pct":88.0,"memory_used_mib":64557.0,"power_w":227.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.489","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":280.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.2491114,"monotonic":2349820.245437813,"hcas":{"mlx5_0":{"port_xmit_data":138731722718207,"port_rcv_data":138763966121905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830990249255,"port_rcv_data":1192111901320,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138843210590864,"port_rcv_data":139149737459924,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2173247025031,"port_rcv_data":1505236484890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.366","gpu_util_pct":67.0,"memory_used_mib":59345.0,"power_w":250.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.380","gpu_util_pct":9.0,"memory_used_mib":59345.0,"power_w":258.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.392","gpu_util_pct":75.0,"memory_used_mib":59345.0,"power_w":259.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.406","gpu_util_pct":49.0,"memory_used_mib":59345.0,"power_w":248.96,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:45.421","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":257.52,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:45.439","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":257.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:45.459","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":262.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.478","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":273.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.24929,"monotonic":2349821.245616916,"hcas":{"mlx5_0":{"port_xmit_data":138731722718472,"port_rcv_data":138763966122118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1830990249255,"port_rcv_data":1192111901320,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138843385146889,"port_rcv_data":139149737756583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2173421571376,"port_rcv_data":1505236770503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.351","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":158.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.363","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":185.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.375","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":190.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.387","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":159.42,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.398","gpu_util_pct":88.0,"memory_used_mib":64557.0,"power_w":254.03,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.408","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":266.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.417","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":261.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.426","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":280.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.2501166,"monotonic":2349822.246443911,"hcas":{"mlx5_0":{"port_xmit_data":138731879555022,"port_rcv_data":138763966286821,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831146292095,"port_rcv_data":1192112065991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138843390680600,"port_rcv_data":139149737775827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2173427091145,"port_rcv_data":1505236789638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.378","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":227.77,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.394","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":196.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.409","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":211.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.422","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":222.95,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.435","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":213.07,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.449","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":158.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.462","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":217.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.474","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":168.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.2508848,"monotonic":2349823.247213164,"hcas":{"mlx5_0":{"port_xmit_data":138731879560357,"port_rcv_data":138763966288664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831146296775,"port_rcv_data":1192112067231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138843680424244,"port_rcv_data":139149738153158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2173716840093,"port_rcv_data":1505237171570,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.361","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":302.39,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.374","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":285.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.386","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":299.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.398","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":305.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:48.409","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":252.77,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.419","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":242.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.429","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":264.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.438","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":270.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.2517138,"monotonic":2349824.248041369,"hcas":{"mlx5_0":{"port_xmit_data":138731956145680,"port_rcv_data":138763966366077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831222881735,"port_rcv_data":1192112144839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138843737320854,"port_rcv_data":139149738275484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2173773713671,"port_rcv_data":1505237302503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.378","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":169.41,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.396","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":164.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.410","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":192.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.424","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":180.74,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.439","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":222.55,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.455","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":219.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.473","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":227.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.493","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":230.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.2524571,"monotonic":2349825.248785008,"hcas":{"mlx5_0":{"port_xmit_data":138732078280026,"port_rcv_data":138763966494910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831344220939,"port_rcv_data":1192112272094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138843911844422,"port_rcv_data":139149738533333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2173948251126,"port_rcv_data":1505237573374,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.375","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":293.67,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.388","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":270.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.399","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":291.27,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.410","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":289.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.420","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":264.99,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.430","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":250.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.440","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":300.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.451","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":267.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.253253,"monotonic":2349826.249581408,"hcas":{"mlx5_0":{"port_xmit_data":138732078286405,"port_rcv_data":138763966496877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831344226791,"port_rcv_data":1192112273458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138844149567363,"port_rcv_data":139149738897685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2174185972220,"port_rcv_data":1505237935550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.366","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":241.7,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.378","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":216.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.390","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":219.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:51.400","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":231.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:51.412","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":290.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.424","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":291.84,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.436","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":288.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.448","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":303.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.254056,"monotonic":2349827.250383468,"hcas":{"mlx5_0":{"port_xmit_data":138732154871760,"port_rcv_data":138763966573155,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831420811751,"port_rcv_data":1192112349284,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138844179591593,"port_rcv_data":139149738984290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2174215958271,"port_rcv_data":1505238031318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.359","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":210.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.374","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":189.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.385","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":180.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.395","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":154.49,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.406","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":176.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.417","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":163.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.428","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":183.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.438","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":174.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.2548661,"monotonic":2349828.251194577,"hcas":{"mlx5_0":{"port_xmit_data":138732169906876,"port_rcv_data":138763966599629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831435052831,"port_rcv_data":1192112373712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138844354152861,"port_rcv_data":139149739278608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2174390515742,"port_rcv_data":1505238322854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.365","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":286.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.386","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":263.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.409","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":289.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.428","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":247.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.450","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":239.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.471","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":227.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.483","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":229.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.495","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":227.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.2556164,"monotonic":2349829.251945439,"hcas":{"mlx5_0":{"port_xmit_data":138732246496878,"port_rcv_data":138763966677392,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831511642471,"port_rcv_data":1192112451683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138844430481959,"port_rcv_data":139149739470859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2174466829906,"port_rcv_data":1505238506434,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.360","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":222.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.373","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":198.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.385","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":216.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.397","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":224.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.407","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":296.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.417","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":268.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.428","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":285.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.438","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":281.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.2557025,"monotonic":2349830.252031033,"hcas":{"mlx5_0":{"port_xmit_data":138732327945496,"port_rcv_data":138763966769204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831592295979,"port_rcv_data":1192112542574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138844605043507,"port_rcv_data":139149739766556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2174641380869,"port_rcv_data":1505238792239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.387","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":245.82,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.401","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":259.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.417","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":275.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.431","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":239.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:55.446","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":237.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.462","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":243.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.477","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":232.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.491","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":248.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.2559729,"monotonic":2349831.252300889,"hcas":{"mlx5_0":{"port_xmit_data":138732404536803,"port_rcv_data":138763966846766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831668886791,"port_rcv_data":1192112619726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138844780236052,"port_rcv_data":139149740069604,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2174816520132,"port_rcv_data":1505239087163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.407","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":253.37,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.419","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":225.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.431","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":253.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.442","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":246.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.453","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":287.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.463","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":292.5,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.473","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":298.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.484","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":299.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.2567542,"monotonic":2349832.25308257,"hcas":{"mlx5_0":{"port_xmit_data":138732481122125,"port_rcv_data":138763966922947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831745471751,"port_rcv_data":1192112695420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138844954770763,"port_rcv_data":139149740342723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2174991030881,"port_rcv_data":1505239336325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.381","gpu_util_pct":0.0,"memory_used_mib":59347.0,"power_w":208.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.394","gpu_util_pct":0.0,"memory_used_mib":59347.0,"power_w":186.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.407","gpu_util_pct":0.0,"memory_used_mib":59347.0,"power_w":188.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.421","gpu_util_pct":0.0,"memory_used_mib":59347.0,"power_w":165.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.434","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":212.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.449","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":212.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.470","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":216.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.488","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":235.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.2575004,"monotonic":2349833.253828536,"hcas":{"mlx5_0":{"port_xmit_data":138732527868120,"port_rcv_data":138763966978729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831791422603,"port_rcv_data":1192112750243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845129323382,"port_rcv_data":139149740630851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175165586942,"port_rcv_data":1505239628664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.361","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.373","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.384","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.395","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:58.405","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":213.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:58.415","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":215.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:58.424","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":211.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:58.433","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":231.68,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.2583246,"monotonic":2349834.254652929,"hcas":{"mlx5_0":{"port_xmit_data":138732527873422,"port_rcv_data":138763966980539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831791427283,"port_rcv_data":1192112751483,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845194982519,"port_rcv_data":139149740710792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175231204204,"port_rcv_data":1505239708449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.382","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.394","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.406","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.419","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.436","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.452","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.470","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.485","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":85.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.2590842,"monotonic":2349835.255413411,"hcas":{"mlx5_0":{"port_xmit_data":138732611715061,"port_rcv_data":138763967074115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831874474007,"port_rcv_data":1192112845150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845194987834,"port_rcv_data":139149740712655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175231208820,"port_rcv_data":1505239709689,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.363","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.375","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.386","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.397","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.406","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.416","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.14,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.425","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.434","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.2599146,"monotonic":2349836.256242945,"hcas":{"mlx5_0":{"port_xmit_data":138732611721375,"port_rcv_data":138763967076017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831874479859,"port_rcv_data":1192112846514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845315596600,"port_rcv_data":139149740840656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175351816888,"port_rcv_data":1505239835921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.365","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":102.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.378","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.391","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.403","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.415","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.428","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.11,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:01.445","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.461","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.2606862,"monotonic":2349837.257015467,"hcas":{"mlx5_0":{"port_xmit_data":138732694366276,"port_rcv_data":138763967167122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831956329975,"port_rcv_data":1192112937040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845315601882,"port_rcv_data":139149740842486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175351821504,"port_rcv_data":1505239837161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.341","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.353","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.364","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.375","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.386","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.395","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.404","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.413","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.261531,"monotonic":2349838.257860199,"hcas":{"mlx5_0":{"port_xmit_data":138732694371579,"port_rcv_data":138763967168932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831956334655,"port_rcv_data":1192112938280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845434416781,"port_rcv_data":139149740968039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175470635832,"port_rcv_data":1505239962029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.362","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.374","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.386","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.397","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.410","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.422","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.434","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.2623281,"monotonic":2349839.258657796,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846378,"port_rcv_data":139149740987224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.364","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.376","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.388","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.401","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.413","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.425","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.437","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.449","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.2631395,"monotonic":2349840.259466095,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.388","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.402","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.452","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:05.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.476","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.486","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.2637098,"monotonic":2349841.260038726,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.383","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.395","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.406","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.417","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.427","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.437","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.264522,"monotonic":2349842.260849808,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.405","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.431","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.443","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.467","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.478","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.489","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.265281,"monotonic":2349843.261609694,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.411","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.423","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.445","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:08.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.474","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.2660635,"monotonic":2349844.262392029,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.400","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.446","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.457","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.468","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.477","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.487","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.2668538,"monotonic":2349845.263182154,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.367","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.379","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.390","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.401","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.411","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.421","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.430","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.440","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.2677057,"monotonic":2349846.264033581,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.385","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.416","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.447","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.470","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.487","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.498","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:11.509","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.519","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.2684493,"monotonic":2349847.264777671,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.349","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.361","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.372","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.382","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:12.392","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:12.402","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.412","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.421","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.2692957,"monotonic":2349848.26562447,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.398","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.416","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.434","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.448","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.459","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.469","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.479","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.490","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.2700455,"monotonic":2349849.266373014,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.371","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.383","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.394","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.405","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.415","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.425","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.434","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.444","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.270879,"monotonic":2349850.267207152,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.371","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.384","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.396","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.412","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.429","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:15.447","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:15.464","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.475","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.2716956,"monotonic":2349851.26802324,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.352","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.365","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.376","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.387","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.397","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.407","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.417","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.426","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.7,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.2725506,"monotonic":2349852.268878613,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.399","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.417","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.433","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.474","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.484","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.2733357,"monotonic":2349853.26966441,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.415","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.427","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.438","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.449","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.459","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.469","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.479","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.488","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.2741196,"monotonic":2349854.270448586,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.378","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.389","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.401","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.412","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.424","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.437","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.449","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.460","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.2749388,"monotonic":2349855.271267194,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.412","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.425","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.436","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.446","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.456","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:20.466","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.475","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.484","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.2757337,"monotonic":2349856.272061816,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.370","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.382","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.394","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.406","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.419","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.431","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.443","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.276546,"monotonic":2349857.272874811,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.419","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.431","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.442","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.453","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.464","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:22.473","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.16,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.492","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.92,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.2773273,"monotonic":2349858.273655716,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.437","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.449","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.460","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.472","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.482","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.492","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.502","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.511","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.2780828,"monotonic":2349859.274412029,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.430","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.441","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.454","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.466","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:24.478","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.489","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.504","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.517","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.2788377,"monotonic":2349860.275166361,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.424","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.437","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.450","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.479","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.497","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.530","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.2795727,"monotonic":2349861.275901926,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.427","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.450","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.461","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:26.471","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.481","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.491","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.280348,"monotonic":2349862.276677421,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.379","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.391","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.402","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.413","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.424","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.434","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.457","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.2804677,"monotonic":2349863.27679508,"hcas":{"mlx5_0":{"port_xmit_data":138732703428408,"port_rcv_data":138763967189360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1831964597119,"port_rcv_data":1192112957128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845442846948,"port_rcv_data":139149740987847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175479063508,"port_rcv_data":1505239980148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.364","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":175.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.377","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":138.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:28.390","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":143.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:28.403","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":133.97,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:28.414","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:28.424","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.434","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.445","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.281288,"monotonic":2349864.277616269,"hcas":{"mlx5_0":{"port_xmit_data":138732780013368,"port_rcv_data":138763967265574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832041182079,"port_rcv_data":1192113032845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845617402828,"port_rcv_data":139149741280076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175653639051,"port_rcv_data":1505240291560,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.382","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":260.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.395","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":308.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.408","gpu_util_pct":5.0,"memory_used_mib":59349.0,"power_w":275.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.422","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":251.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.435","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":184.93,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.451","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":189.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.470","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":201.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.490","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":194.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.2820513,"monotonic":2349865.278379568,"hcas":{"mlx5_0":{"port_xmit_data":138732780013368,"port_rcv_data":138763967265574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832041182079,"port_rcv_data":1192113032845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845676288982,"port_rcv_data":139149741436193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175712518115,"port_rcv_data":1505240453321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.365","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":85.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.377","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":110.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.389","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":88.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.399","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":83.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:30.409","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":225.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:30.419","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":225.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.429","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":239.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.438","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":212.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.2828875,"monotonic":2349866.279216064,"hcas":{"mlx5_0":{"port_xmit_data":138732780013368,"port_rcv_data":138763967265574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832041182079,"port_rcv_data":1192113032845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845676288982,"port_rcv_data":139149741436193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175712518115,"port_rcv_data":1505240453321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.373","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.385","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.396","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.408","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.419","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":86.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.431","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":105.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.443","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.454","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":97.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.283694,"monotonic":2349867.280023467,"hcas":{"mlx5_0":{"port_xmit_data":138732780013368,"port_rcv_data":138763967265574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832041182079,"port_rcv_data":1192113032845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845676288982,"port_rcv_data":139149741436193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175712518115,"port_rcv_data":1505240453321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.384","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.396","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.407","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:32.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:32.428","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.437","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.445","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.453","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.284206,"monotonic":2349868.280533527,"hcas":{"mlx5_0":{"port_xmit_data":138732780013368,"port_rcv_data":138763967265574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832041182079,"port_rcv_data":1192113032845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845676288982,"port_rcv_data":139149741436193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175712518115,"port_rcv_data":1505240453321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.373","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.389","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.405","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.423","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.438","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.450","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.461","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.471","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.285013,"monotonic":2349869.281342182,"hcas":{"mlx5_0":{"port_xmit_data":138732780013633,"port_rcv_data":138763967265787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832041182079,"port_rcv_data":1192113032845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845676288982,"port_rcv_data":139149741436193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175712518115,"port_rcv_data":1505240453321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.387","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.399","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.411","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.422","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:34.433","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.443","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.452","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.462","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.2858293,"monotonic":2349870.282158219,"hcas":{"mlx5_0":{"port_xmit_data":138732780013848,"port_rcv_data":138763967265963,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832041182079,"port_rcv_data":1192113032845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845676288982,"port_rcv_data":139149741436193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175712518115,"port_rcv_data":1505240453321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.433","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.445","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.460","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.475","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.485","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.495","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.517","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.2866015,"monotonic":2349871.28292941,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.387","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.399","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.411","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.422","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.432","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.442","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.452","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.462","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.2874265,"monotonic":2349872.283755565,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.396","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.413","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.424","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.456","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.475","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.2877076,"monotonic":2349873.284035773,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.428","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.450","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:38.461","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:38.471","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.480","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.490","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.499","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.288511,"monotonic":2349874.284839524,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.428","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.441","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.454","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.467","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.479","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.495","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.37,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.512","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.530","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.2892578,"monotonic":2349875.285585841,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.446","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.458","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.478","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.488","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.497","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.2900362,"monotonic":2349876.286365174,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.407","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.419","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:41.431","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:41.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.456","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.469","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.480","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.491","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.2908237,"monotonic":2349877.287152564,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.447","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.458","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.469","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.479","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.488","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.498","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.507","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.2915995,"monotonic":2349878.287928613,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.440","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.454","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.467","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:43.479","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:43.496","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.532","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.547","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.291701,"monotonic":2349879.288029928,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.451","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.493","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.504","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.515","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.292468,"monotonic":2349880.288796979,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.454","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.470","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.487","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.505","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.518","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.528","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.539","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.2932088,"monotonic":2349881.289538479,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.433","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.452","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.475","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.485","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.495","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.505","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.294,"monotonic":2349882.290327778,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.446","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.465","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.477","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.488","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.498","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.508","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.517","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.527","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.2947588,"monotonic":2349883.291088205,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.436","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.448","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.464","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:48.487","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:48.509","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.531","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.548","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.559","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.2954865,"monotonic":2349884.291815424,"hcas":{"mlx5_0":{"port_xmit_data":138732863860964,"port_rcv_data":138763967360089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832124234655,"port_rcv_data":1192113126698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845796902543,"port_rcv_data":139149741566002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2175833130799,"port_rcv_data":1505240582498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.406","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":222.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.425","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":135.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:49.442","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":144.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.465","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":200.3,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.478","gpu_util_pct":35.0,"memory_used_mib":64563.0,"power_w":95.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.490","gpu_util_pct":10.0,"memory_used_mib":64563.0,"power_w":79.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.501","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":104.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.512","gpu_util_pct":27.0,"memory_used_mib":64563.0,"power_w":112.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.296266,"monotonic":2349885.292594346,"hcas":{"mlx5_0":{"port_xmit_data":138732940445924,"port_rcv_data":138763967436969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832200819615,"port_rcv_data":1192113203454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138845971472528,"port_rcv_data":139149741872801,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176007702177,"port_rcv_data":1505240890705,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.397","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":264.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.409","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":264.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.420","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":273.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.432","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":275.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.442","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":321.9,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.453","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":292.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.463","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":321.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.475","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":307.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.297084,"monotonic":2349886.293412818,"hcas":{"mlx5_0":{"port_xmit_data":138732940445924,"port_rcv_data":138763967436969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832200819615,"port_rcv_data":1192113203454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846030392529,"port_rcv_data":139149742037500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176066610442,"port_rcv_data":1505241063030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.420","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.434","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.449","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":103.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.464","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":100.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.481","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":205.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.499","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":175.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.512","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":203.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.523","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":183.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.2978578,"monotonic":2349887.29418635,"hcas":{"mlx5_0":{"port_xmit_data":138732940445924,"port_rcv_data":138763967436969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832200819615,"port_rcv_data":1192113203454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846030392529,"port_rcv_data":139149742037500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176066610442,"port_rcv_data":1505241063030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.417","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.429","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.57,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.440","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.451","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.461","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.470","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":103.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.480","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":98.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.489","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":111.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.2986698,"monotonic":2349888.294998574,"hcas":{"mlx5_0":{"port_xmit_data":138732940445924,"port_rcv_data":138763967436969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832200819615,"port_rcv_data":1192113203454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846030392529,"port_rcv_data":139149742037500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176066610442,"port_rcv_data":1505241063030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.425","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.455","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.470","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.482","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.494","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.511","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.528","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.2994285,"monotonic":2349889.295756924,"hcas":{"mlx5_0":{"port_xmit_data":138732940445924,"port_rcv_data":138763967436969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832200819615,"port_rcv_data":1192113203454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846030392529,"port_rcv_data":139149742037500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176066610442,"port_rcv_data":1505241063030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.402","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.414","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:54.425","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.436","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.456","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.474","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.299707,"monotonic":2349890.296036011,"hcas":{"mlx5_0":{"port_xmit_data":138732940446189,"port_rcv_data":138763967437182,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832200819615,"port_rcv_data":1192113203454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846030392529,"port_rcv_data":139149742037500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176066610442,"port_rcv_data":1505241063030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.403","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.414","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.425","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.436","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.445","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.464","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.473","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.3005474,"monotonic":2349891.296875684,"hcas":{"mlx5_0":{"port_xmit_data":138733024293422,"port_rcv_data":138763967530566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832283872191,"port_rcv_data":1192113295617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846151006122,"port_rcv_data":139149742168039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176187223126,"port_rcv_data":1505241191618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.401","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.413","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.424","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.456","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.475","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.3013542,"monotonic":2349892.297682183,"hcas":{"mlx5_0":{"port_xmit_data":138733024293422,"port_rcv_data":138763967530566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832283872191,"port_rcv_data":1192113295617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846151006122,"port_rcv_data":139149742168039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176187223126,"port_rcv_data":1505241191618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.405","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.14,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.430","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.447","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.462","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.480","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.496","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.508","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.3021388,"monotonic":2349893.298467261,"hcas":{"mlx5_0":{"port_xmit_data":138733024293422,"port_rcv_data":138763967530566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832283872191,"port_rcv_data":1192113295617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846151006122,"port_rcv_data":139149742168039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176187223126,"port_rcv_data":1505241191618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.403","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.414","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.57,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.425","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.436","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.456","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.474","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.302965,"monotonic":2349894.299293946,"hcas":{"mlx5_0":{"port_xmit_data":138733024293422,"port_rcv_data":138763967530566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832283872191,"port_rcv_data":1192113295617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846151006122,"port_rcv_data":139149742168039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176187223126,"port_rcv_data":1505241191618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.428","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.452","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.516","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.528","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.303736,"monotonic":2349895.300064261,"hcas":{"mlx5_0":{"port_xmit_data":138733024293422,"port_rcv_data":138763967530566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832283872191,"port_rcv_data":1192113295617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846151006122,"port_rcv_data":139149742168039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176187223126,"port_rcv_data":1505241191618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.403","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":93.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.415","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.426","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.436","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.446","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.464","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.473","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.3045702,"monotonic":2349896.300897782,"hcas":{"mlx5_0":{"port_xmit_data":138733024293422,"port_rcv_data":138763967530566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832283872191,"port_rcv_data":1192113295617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846151006122,"port_rcv_data":139149742168039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176187223126,"port_rcv_data":1505241191618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.452","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":108.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.92,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.475","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:01.491","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.507","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.524","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.540","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.551","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.3053167,"monotonic":2349897.301645029,"hcas":{"mlx5_0":{"port_xmit_data":138733100878382,"port_rcv_data":138763967606578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832360457151,"port_rcv_data":1192113371722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846151006122,"port_rcv_data":139149742168039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176187223126,"port_rcv_data":1505241191618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.414","gpu_util_pct":97.0,"memory_used_mib":59349.0,"power_w":229.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.427","gpu_util_pct":96.0,"memory_used_mib":59349.0,"power_w":201.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.438","gpu_util_pct":97.0,"memory_used_mib":59349.0,"power_w":210.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.449","gpu_util_pct":97.0,"memory_used_mib":59349.0,"power_w":189.21,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.460","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":110.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.471","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":107.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.481","gpu_util_pct":95.0,"memory_used_mib":64563.0,"power_w":124.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.491","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":115.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.3061335,"monotonic":2349898.302461073,"hcas":{"mlx5_0":{"port_xmit_data":138733100878382,"port_rcv_data":138763967606578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832360457151,"port_rcv_data":1192113371722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846325557178,"port_rcv_data":139149742455429,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176361784921,"port_rcv_data":1505241490041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.393","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":229.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.407","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":256.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.420","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":234.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:03.433","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":212.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.448","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":296.26,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.465","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":306.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.484","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":303.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.504","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":303.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.306921,"monotonic":2349899.303250274,"hcas":{"mlx5_0":{"port_xmit_data":138733100878382,"port_rcv_data":138763967606578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832360457151,"port_rcv_data":1192113371722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846383735373,"port_rcv_data":139149742609555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176419933103,"port_rcv_data":1505241652988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.387","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":110.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.399","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":108.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.410","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":84.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.420","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":98.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.431","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":176.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.440","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":168.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.449","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":155.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.458","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":160.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.3077192,"monotonic":2349900.304046664,"hcas":{"mlx5_0":{"port_xmit_data":138733100878382,"port_rcv_data":138763967606578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832360457151,"port_rcv_data":1192113371722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846383735373,"port_rcv_data":139149742609555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176419933103,"port_rcv_data":1505241652988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.443","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.455","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.466","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.477","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.487","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":102.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.496","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":91.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":107.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.515","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":85.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.3085163,"monotonic":2349901.304844826,"hcas":{"mlx5_0":{"port_xmit_data":138733100878382,"port_rcv_data":138763967606578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832360457151,"port_rcv_data":1192113371722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846383735373,"port_rcv_data":139149742609555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176419933103,"port_rcv_data":1505241652988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.410","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.422","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.434","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.464","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.474","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.3093352,"monotonic":2349902.305664299,"hcas":{"mlx5_0":{"port_xmit_data":138733100878382,"port_rcv_data":138763967606578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832360457151,"port_rcv_data":1192113371722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846383735373,"port_rcv_data":139149742609555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176419933103,"port_rcv_data":1505241652988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.455","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.466","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.477","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.487","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.496","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.310121,"monotonic":2349903.306450416,"hcas":{"mlx5_0":{"port_xmit_data":138733100878744,"port_rcv_data":138763967606888,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832360457151,"port_rcv_data":1192113371722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846383735373,"port_rcv_data":139149742609555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176419933103,"port_rcv_data":1505241652988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.390","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.402","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.413","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.423","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.433","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.442","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.451","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.460","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.3109832,"monotonic":2349904.307310891,"hcas":{"mlx5_0":{"port_xmit_data":138733183528455,"port_rcv_data":138763967700324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832442311947,"port_rcv_data":1192113465140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846502555131,"port_rcv_data":139149742739119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176538752047,"port_rcv_data":1505241780196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.432","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.445","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.33,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:09.458","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:09.471","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.517","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.3116994,"monotonic":2349905.308028688,"hcas":{"mlx5_0":{"port_xmit_data":138733183528455,"port_rcv_data":138763967700324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832442311947,"port_rcv_data":1192113465140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846502555131,"port_rcv_data":139149742739119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176538752047,"port_rcv_data":1505241780196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.447","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.458","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.469","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.480","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.489","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.498","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.507","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.312498,"monotonic":2349906.308827612,"hcas":{"mlx5_0":{"port_xmit_data":138733183528455,"port_rcv_data":138763967700324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832442311947,"port_rcv_data":1192113465140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846502555131,"port_rcv_data":139149742739119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176538752047,"port_rcv_data":1505241780196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.02,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.474","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.486","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.496","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.505","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.515","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.524","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.3132782,"monotonic":2349907.309607761,"hcas":{"mlx5_0":{"port_xmit_data":138733183528455,"port_rcv_data":138763967700324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832442311947,"port_rcv_data":1192113465140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846502555131,"port_rcv_data":139149742739119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176538752047,"port_rcv_data":1505241780196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.446","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.457","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.478","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.488","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.497","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.314083,"monotonic":2349908.31041277,"hcas":{"mlx5_0":{"port_xmit_data":138733183528455,"port_rcv_data":138763967700324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832442311947,"port_rcv_data":1192113465140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846502555131,"port_rcv_data":139149742739119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176538752047,"port_rcv_data":1505241780196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":112.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.475","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.26,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:13.487","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.499","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.512","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.524","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.536","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.548","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.3148391,"monotonic":2349909.31116826,"hcas":{"mlx5_0":{"port_xmit_data":138733183528455,"port_rcv_data":138763967700324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832442311947,"port_rcv_data":1192113465140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846502555131,"port_rcv_data":139149742739119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176538752047,"port_rcv_data":1505241780196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":117.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.474","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.33,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.485","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.495","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.505","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.4,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.523","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.532","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.3150067,"monotonic":2349910.311334041,"hcas":{"mlx5_0":{"port_xmit_data":138733260113415,"port_rcv_data":138763967776569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832518896907,"port_rcv_data":1192113541493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846502555131,"port_rcv_data":139149742739119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176538752047,"port_rcv_data":1505241780196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.454","gpu_util_pct":96.0,"memory_used_mib":59349.0,"power_w":229.69,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.469","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":199.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.485","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":199.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.499","gpu_util_pct":99.0,"memory_used_mib":59349.0,"power_w":202.57,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.514","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":115.94,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.531","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":123.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.549","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":137.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.567","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":133.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.3157423,"monotonic":2349911.312070442,"hcas":{"mlx5_0":{"port_xmit_data":138733260113415,"port_rcv_data":138763967776569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832518896907,"port_rcv_data":1192113541493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846677110379,"port_rcv_data":139149743031662,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176713301316,"port_rcv_data":1505242067799,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":207.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:16.430","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":217.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:16.441","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":203.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.452","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":209.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.463","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":296.33,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.473","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":323.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.484","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":313.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.495","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":301.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.3165562,"monotonic":2349912.312885341,"hcas":{"mlx5_0":{"port_xmit_data":138733260113415,"port_rcv_data":138763967776569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832518896907,"port_rcv_data":1192113541493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846734271244,"port_rcv_data":139149743189011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176770422139,"port_rcv_data":1505242216704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.447","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":85.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.459","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.470","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":91.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.482","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":112.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.492","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":128.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.502","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":143.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.512","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":127.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.521","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":116.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.31735,"monotonic":2349913.313679123,"hcas":{"mlx5_0":{"port_xmit_data":138733260113415,"port_rcv_data":138763967776569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832518896907,"port_rcv_data":1192113541493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846734271244,"port_rcv_data":139149743189011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176770422139,"port_rcv_data":1505242216704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.397","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.409","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:18.421","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.432","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.442","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.452","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.461","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":102.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.470","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":86.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.3181918,"monotonic":2349914.314521126,"hcas":{"mlx5_0":{"port_xmit_data":138733260113415,"port_rcv_data":138763967776569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832518896907,"port_rcv_data":1192113541493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846734271244,"port_rcv_data":139149743189011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176770422139,"port_rcv_data":1505242216704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.450","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.476","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.493","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.509","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.527","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.543","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.557","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.3189435,"monotonic":2349915.315272642,"hcas":{"mlx5_0":{"port_xmit_data":138733260113415,"port_rcv_data":138763967776569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832518896907,"port_rcv_data":1192113541493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846734271244,"port_rcv_data":139149743189011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176770422139,"port_rcv_data":1505242216704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:20.451","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:20.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.474","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.484","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.494","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.504","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.3197465,"monotonic":2349916.316075558,"hcas":{"mlx5_0":{"port_xmit_data":138733260113810,"port_rcv_data":138763967776912,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832518896907,"port_rcv_data":1192113541493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846734271244,"port_rcv_data":139149743189011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176770422139,"port_rcv_data":1505242216704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.409","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.422","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.433","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.459","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.480","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.504","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.530","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.3205366,"monotonic":2349917.316864264,"hcas":{"mlx5_0":{"port_xmit_data":138733341567925,"port_rcv_data":138763967868393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832599556267,"port_rcv_data":1192113633563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846851295079,"port_rcv_data":139149743314205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176887444999,"port_rcv_data":1505242340255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.438","gpu_util_pct":56.0,"memory_used_mib":59349.0,"power_w":139.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.452","gpu_util_pct":98.0,"memory_used_mib":59349.0,"power_w":110.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.464","gpu_util_pct":50.0,"memory_used_mib":59349.0,"power_w":135.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.476","gpu_util_pct":73.0,"memory_used_mib":59349.0,"power_w":138.38,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.486","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.496","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.505","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.515","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.3213465,"monotonic":2349918.317675252,"hcas":{"mlx5_0":{"port_xmit_data":138733418152885,"port_rcv_data":138763967944297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832676141227,"port_rcv_data":1192113709498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138846851295079,"port_rcv_data":139149743314205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2176887444999,"port_rcv_data":1505242340255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.441","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":341.15,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.453","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":357.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.467","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":332.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.480","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":337.52,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.494","gpu_util_pct":67.0,"memory_used_mib":64563.0,"power_w":239.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.507","gpu_util_pct":35.0,"memory_used_mib":64563.0,"power_w":248.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.522","gpu_util_pct":8.0,"memory_used_mib":64563.0,"power_w":262.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.536","gpu_util_pct":59.0,"memory_used_mib":64563.0,"power_w":251.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.3220727,"monotonic":2349919.318400432,"hcas":{"mlx5_0":{"port_xmit_data":138733418152885,"port_rcv_data":138763967944297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832676141227,"port_rcv_data":1192113709498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847082905553,"port_rcv_data":139149743766059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177119031784,"port_rcv_data":1505242787471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.424","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":106.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.436","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":112.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.448","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":106.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.458","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":100.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.469","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":263.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.480","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":270.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.492","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":269.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.503","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":253.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.3228917,"monotonic":2349920.319219336,"hcas":{"mlx5_0":{"port_xmit_data":138733418152885,"port_rcv_data":138763967944297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832676141227,"port_rcv_data":1192113709498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847082905553,"port_rcv_data":139149743766059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177119031784,"port_rcv_data":1505242787471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.454","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.467","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.479","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.489","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":103.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":100.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.510","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":90.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.519","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.3237066,"monotonic":2349921.320035092,"hcas":{"mlx5_0":{"port_xmit_data":138733418152885,"port_rcv_data":138763967944297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832676141227,"port_rcv_data":1192113709498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847082905553,"port_rcv_data":139149743766059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177119031784,"port_rcv_data":1505242787471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.406","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.430","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.441","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.452","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.462","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.471","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.481","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.32456,"monotonic":2349922.320887447,"hcas":{"mlx5_0":{"port_xmit_data":138733418152885,"port_rcv_data":138763967944297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832676141227,"port_rcv_data":1192113709498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847082905553,"port_rcv_data":139149743766059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177119031784,"port_rcv_data":1505242787471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.406","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":125.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.418","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.431","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.448","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.463","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.482","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.498","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.510","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.32538,"monotonic":2349923.321707599,"hcas":{"mlx5_0":{"port_xmit_data":138733418153280,"port_rcv_data":138763967944640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832676141227,"port_rcv_data":1192113709498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847082905553,"port_rcv_data":139149743766059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177119031784,"port_rcv_data":1505242787471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.427","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.451","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:28.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.473","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.16,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.493","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.502","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.3262122,"monotonic":2349924.322539491,"hcas":{"mlx5_0":{"port_xmit_data":138733418153495,"port_rcv_data":138763967944816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832676141227,"port_rcv_data":1192113709498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847082905553,"port_rcv_data":139149743766059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177119031784,"port_rcv_data":1505242787471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.421","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.440","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.453","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.464","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.473","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.482","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.490","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.498","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.3270316,"monotonic":2349925.323359455,"hcas":{"mlx5_0":{"port_xmit_data":138733576192354,"port_rcv_data":138763968113354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832833385547,"port_rcv_data":1192113878609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847199929322,"port_rcv_data":139149743892506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177236054644,"port_rcv_data":1505242912106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.434","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":315.6,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.446","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":258.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.457","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":323.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.469","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":337.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.480","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":218.38,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.490","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":212.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.499","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":197.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.508","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":224.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.3278334,"monotonic":2349926.32416237,"hcas":{"mlx5_0":{"port_xmit_data":138733576192354,"port_rcv_data":138763968113354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832833385547,"port_rcv_data":1192113878609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847374474380,"port_rcv_data":139149744174409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177410608979,"port_rcv_data":1505243203922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.412","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":144.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.428","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":171.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":156.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.461","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":112.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.477","gpu_util_pct":79.0,"memory_used_mib":64563.0,"power_w":273.73,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.488","gpu_util_pct":96.0,"memory_used_mib":64563.0,"power_w":281.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.499","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":292.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.509","gpu_util_pct":87.0,"memory_used_mib":64563.0,"power_w":276.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.3286362,"monotonic":2349927.324965253,"hcas":{"mlx5_0":{"port_xmit_data":138733576192354,"port_rcv_data":138763968113354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832833385547,"port_rcv_data":1192113878609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847433292824,"port_rcv_data":139149744338394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177469393268,"port_rcv_data":1505243370864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.409","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":114.31,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.421","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":102.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:32.432","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":112.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:32.443","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":109.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.453","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":101.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.462","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":86.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.471","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":104.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.481","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.3294723,"monotonic":2349928.325800834,"hcas":{"mlx5_0":{"port_xmit_data":138733576192354,"port_rcv_data":138763968113354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832833385547,"port_rcv_data":1192113878609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847433292824,"port_rcv_data":139149744338394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177469393268,"port_rcv_data":1505243370864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.411","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.425","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.443","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.454","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.471","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.488","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.505","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.520","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.33026,"monotonic":2349929.326588591,"hcas":{"mlx5_0":{"port_xmit_data":138733576192354,"port_rcv_data":138763968113354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832833385547,"port_rcv_data":1192113878609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847433292824,"port_rcv_data":139149744338394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177469393268,"port_rcv_data":1505243370864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.446","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.457","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.478","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.488","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.497","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.3310692,"monotonic":2349930.327399243,"hcas":{"mlx5_0":{"port_xmit_data":138733576192354,"port_rcv_data":138763968113354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832833385547,"port_rcv_data":1192113878609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847433292824,"port_rcv_data":139149744338394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177469393268,"port_rcv_data":1505243370864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.421","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.430","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:35.440","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:35.449","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.459","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.467","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.476","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.485","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.3319037,"monotonic":2349931.328233677,"hcas":{"mlx5_0":{"port_xmit_data":138733576192554,"port_rcv_data":138763968113502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832833385547,"port_rcv_data":1192113878609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847433292824,"port_rcv_data":139149744338394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177469393268,"port_rcv_data":1505243370864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.412","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.424","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.445","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.455","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.465","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.474","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.483","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.3327382,"monotonic":2349932.329066953,"hcas":{"mlx5_0":{"port_xmit_data":138733660034453,"port_rcv_data":138763968209275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832916432271,"port_rcv_data":1192113972058,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847433297716,"port_rcv_data":139149744339834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177469397884,"port_rcv_data":1505243372104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.462","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":211.6,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.481","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":218.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:37.499","gpu_util_pct":93.0,"memory_used_mib":59349.0,"power_w":253.42,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.522","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":243.4,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.537","gpu_util_pct":51.0,"memory_used_mib":64563.0,"power_w":108.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.549","gpu_util_pct":20.0,"memory_used_mib":64563.0,"power_w":112.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.560","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":103.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.570","gpu_util_pct":43.0,"memory_used_mib":64563.0,"power_w":141.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.3334787,"monotonic":2349933.329807884,"hcas":{"mlx5_0":{"port_xmit_data":138733736625137,"port_rcv_data":138763968286205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832993023083,"port_rcv_data":1192114049425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847728449330,"port_rcv_data":139149744746816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177764562814,"port_rcv_data":1505243793147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.453","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":247.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.465","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":248.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.476","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":252.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.487","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":223.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.497","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":303.62,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.508","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":331.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.519","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":304.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.530","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":311.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.334247,"monotonic":2349934.330576764,"hcas":{"mlx5_0":{"port_xmit_data":138733736625137,"port_rcv_data":138763968286205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832993023083,"port_rcv_data":1192114049425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847784935108,"port_rcv_data":139149744902130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177821025162,"port_rcv_data":1505243947075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":94.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.452","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":104.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":97.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.484","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":106.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.502","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":177.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.517","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":163.72,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.528","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":161.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.539","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":165.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.3350263,"monotonic":2349935.331355324,"hcas":{"mlx5_0":{"port_xmit_data":138733736625137,"port_rcv_data":138763968286205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832993023083,"port_rcv_data":1192114049425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847784935108,"port_rcv_data":139149744902130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177821025162,"port_rcv_data":1505243947075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.475","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.486","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.496","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.516","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":93.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.526","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":101.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":86.56,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.3357975,"monotonic":2349936.332127106,"hcas":{"mlx5_0":{"port_xmit_data":138733736625137,"port_rcv_data":138763968286205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832993023083,"port_rcv_data":1192114049425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847784935108,"port_rcv_data":139149744902130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177821025162,"port_rcv_data":1505243947075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.417","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.432","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.449","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.466","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.479","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.490","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.510","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.3365917,"monotonic":2349937.332921078,"hcas":{"mlx5_0":{"port_xmit_data":138733736625137,"port_rcv_data":138763968286205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832993023083,"port_rcv_data":1192114049425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847784935108,"port_rcv_data":139149744902130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177821025162,"port_rcv_data":1505243947075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.439","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:42.451","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:42.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.482","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.492","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.501","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.510","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.337403,"monotonic":2349938.333732832,"hcas":{"mlx5_0":{"port_xmit_data":138733736625500,"port_rcv_data":138763968286515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1832993023083,"port_rcv_data":1192114049425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847784935108,"port_rcv_data":139149744902130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177821025162,"port_rcv_data":1505243947075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.456","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.479","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.489","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.509","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.518","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.528","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.61,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.3382003,"monotonic":2349939.334529888,"hcas":{"mlx5_0":{"port_xmit_data":138733817480041,"port_rcv_data":138763968378008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833073082839,"port_rcv_data":1192114138550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847901062273,"port_rcv_data":139149745025368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177937151546,"port_rcv_data":1505244070858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.465","gpu_util_pct":63.0,"memory_used_mib":59349.0,"power_w":133.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.479","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":154.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.491","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":154.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.504","gpu_util_pct":82.0,"memory_used_mib":59349.0,"power_w":165.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.515","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.525","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.544","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.3389766,"monotonic":2349940.335305615,"hcas":{"mlx5_0":{"port_xmit_data":138733894065001,"port_rcv_data":138763968455864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833149667799,"port_rcv_data":1192114215833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138847901062273,"port_rcv_data":139149745025368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2177937151546,"port_rcv_data":1505244070858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:45.489","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":349.35,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:45.502","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":324.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:45.514","gpu_util_pct":91.0,"memory_used_mib":59349.0,"power_w":347.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.525","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":362.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.536","gpu_util_pct":55.0,"memory_used_mib":64563.0,"power_w":250.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.546","gpu_util_pct":23.0,"memory_used_mib":64563.0,"power_w":254.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.557","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":257.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.567","gpu_util_pct":47.0,"memory_used_mib":64563.0,"power_w":269.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.3397317,"monotonic":2349941.336061096,"hcas":{"mlx5_0":{"port_xmit_data":138733970649961,"port_rcv_data":138763968531203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833226252759,"port_rcv_data":1192114291460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848134703166,"port_rcv_data":139149745473894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178170803080,"port_rcv_data":1505244533857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.461","gpu_util_pct":84.0,"memory_used_mib":59349.0,"power_w":269.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.473","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":282.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.487","gpu_util_pct":76.0,"memory_used_mib":59349.0,"power_w":275.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.500","gpu_util_pct":98.0,"memory_used_mib":59349.0,"power_w":273.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.511","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":343.27,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.522","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":327.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.532","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":350.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.542","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":357.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.3405356,"monotonic":2349942.33686389,"hcas":{"mlx5_0":{"port_xmit_data":138733970649961,"port_rcv_data":138763968531203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833226252759,"port_rcv_data":1192114291460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848309221630,"port_rcv_data":139149745730150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178345346902,"port_rcv_data":1505244814804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.454","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":198.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.466","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":221.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.477","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":211.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.488","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":180.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.498","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":290.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.511","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":299.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.522","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":276.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.532","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":284.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.3413393,"monotonic":2349943.33766772,"hcas":{"mlx5_0":{"port_xmit_data":138733970649961,"port_rcv_data":138763968531203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833226252759,"port_rcv_data":1192114291460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848367590450,"port_rcv_data":139149745876191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178403709269,"port_rcv_data":1505244970807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.448","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.461","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.472","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.483","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.494","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":159.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.504","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":178.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.513","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":137.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.523","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":168.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.3421485,"monotonic":2349944.338477923,"hcas":{"mlx5_0":{"port_xmit_data":138733970649961,"port_rcv_data":138763968531203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833226252759,"port_rcv_data":1192114291460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848367590450,"port_rcv_data":139149745876191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178403709269,"port_rcv_data":1505244970807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.435","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.447","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.459","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.470","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.481","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.491","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.501","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":86.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.510","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.3429787,"monotonic":2349945.339308013,"hcas":{"mlx5_0":{"port_xmit_data":138733970650226,"port_rcv_data":138763968531416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833226252759,"port_rcv_data":1192114291460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848367590450,"port_rcv_data":139149745876191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178403709269,"port_rcv_data":1505244970807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.447","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.459","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.470","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.480","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.490","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.509","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.519","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.343797,"monotonic":2349946.340126403,"hcas":{"mlx5_0":{"port_xmit_data":138733970650226,"port_rcv_data":138763968531416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833226252759,"port_rcv_data":1192114291460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848367590450,"port_rcv_data":139149745876191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178403709269,"port_rcv_data":1505244970807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.457","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.469","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.480","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.491","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.501","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.511","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.521","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.530","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.3446093,"monotonic":2349947.340937944,"hcas":{"mlx5_0":{"port_xmit_data":138734054491735,"port_rcv_data":138763968623854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833309299483,"port_rcv_data":1192114381515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848367595342,"port_rcv_data":139149745877631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178403713885,"port_rcv_data":1505244972047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:52.490","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":114.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:52.502","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.514","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.545","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.555","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.565","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.345388,"monotonic":2349948.341716319,"hcas":{"mlx5_0":{"port_xmit_data":138734054498114,"port_rcv_data":138763968625821,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833309305335,"port_rcv_data":1192114382879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848488204044,"port_rcv_data":139149746005908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178524321953,"port_rcv_data":1505245100650,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.456","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.479","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.491","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.501","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.511","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.521","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.531","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.3462133,"monotonic":2349949.342541258,"hcas":{"mlx5_0":{"port_xmit_data":138734214331025,"port_rcv_data":138763968794807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833468344503,"port_rcv_data":1192114551658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848607920784,"port_rcv_data":139149746134634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178644037245,"port_rcv_data":1505245230618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.482","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":279.03,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.495","gpu_util_pct":95.0,"memory_used_mib":59349.0,"power_w":252.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.509","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":259.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.521","gpu_util_pct":98.0,"memory_used_mib":59349.0,"power_w":254.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.533","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":139.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.544","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":123.72,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.555","gpu_util_pct":55.0,"memory_used_mib":64563.0,"power_w":135.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.566","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":155.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.346753,"monotonic":2349950.343081206,"hcas":{"mlx5_0":{"port_xmit_data":138734214331025,"port_rcv_data":138763968794807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833468344503,"port_rcv_data":1192114551658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848782460822,"port_rcv_data":139149746410981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178818587217,"port_rcv_data":1505245518629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.465","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":258.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.482","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":222.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.495","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":230.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.508","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":197.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.521","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":308.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.534","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":306.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.548","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":318.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.561","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":325.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.3475227,"monotonic":2349951.34385165,"hcas":{"mlx5_0":{"port_xmit_data":138734214331025,"port_rcv_data":138763968794807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833468344503,"port_rcv_data":1192114551658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848841607710,"port_rcv_data":139149746572163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178877700911,"port_rcv_data":1505245679358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:56.452","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":113.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.464","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":104.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.475","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":109.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.491","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":93.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":172.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.536","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":155.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.560","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":173.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.574","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":144.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.347712,"monotonic":2349952.344041276,"hcas":{"mlx5_0":{"port_xmit_data":138734214331025,"port_rcv_data":138763968794807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833468344503,"port_rcv_data":1192114551658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848841607710,"port_rcv_data":139149746572163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178877700911,"port_rcv_data":1505245679358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.449","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.461","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.485","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.501","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":98.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.518","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.536","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.551","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":114.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.3485034,"monotonic":2349953.344832157,"hcas":{"mlx5_0":{"port_xmit_data":138734214331025,"port_rcv_data":138763968794807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833468344503,"port_rcv_data":1192114551658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848841607710,"port_rcv_data":139149746572163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178877700911,"port_rcv_data":1505245679358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:58.450","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.484","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.494","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.504","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.513","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.522","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.3493278,"monotonic":2349954.345657041,"hcas":{"mlx5_0":{"port_xmit_data":138734214331025,"port_rcv_data":138763968794807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833468344503,"port_rcv_data":1192114551658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848841607710,"port_rcv_data":139149746572163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178877700911,"port_rcv_data":1505245679358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.464","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.476","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.487","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.497","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.507","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.516","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.525","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.534","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.3501422,"monotonic":2349955.346471471,"hcas":{"mlx5_0":{"port_xmit_data":138734214331420,"port_rcv_data":138763968795150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833468344503,"port_rcv_data":1192114551658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848841607710,"port_rcv_data":139149746572163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178877700911,"port_rcv_data":1505245679358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.452","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":155.91,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.464","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":144.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.476","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":146.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.488","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":173.87,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.499","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.508","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.517","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.526","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.3509638,"monotonic":2349956.347292645,"hcas":{"mlx5_0":{"port_xmit_data":138734374758311,"port_rcv_data":138763968966069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833627976187,"port_rcv_data":1192114722623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138848841612454,"port_rcv_data":139149746573403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2178877705527,"port_rcv_data":1505245680598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:01.460","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":189.22,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.476","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":200.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.494","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":221.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.516","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":204.15,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.532","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":144.62,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.544","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":149.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.556","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":141.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.566","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":160.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.3517148,"monotonic":2349957.348042813,"hcas":{"mlx5_0":{"port_xmit_data":138734374764035,"port_rcv_data":138763968967433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833627982039,"port_rcv_data":1192114723987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849136768488,"port_rcv_data":139149746985983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179172830637,"port_rcv_data":1505246064593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.453","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":179.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.465","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":166.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.476","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":189.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.487","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":146.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.497","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":294.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.507","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":305.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.516","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":297.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.527","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":296.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.3525443,"monotonic":2349958.348872806,"hcas":{"mlx5_0":{"port_xmit_data":138734374764035,"port_rcv_data":138763968967433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833627982039,"port_rcv_data":1192114723987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849195457225,"port_rcv_data":139149747145413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179231514202,"port_rcv_data":1505246225052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.471","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":130.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.483","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.495","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.505","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.516","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":110.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.525","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":96.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":102.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.544","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":87.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.3533614,"monotonic":2349959.349689663,"hcas":{"mlx5_0":{"port_xmit_data":138734374764035,"port_rcv_data":138763968967433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833627982039,"port_rcv_data":1192114723987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849195457225,"port_rcv_data":139149747145413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179231514202,"port_rcv_data":1505246225052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.455","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":130.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.469","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.480","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.490","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.510","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.519","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.528","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.3541925,"monotonic":2349960.35052112,"hcas":{"mlx5_0":{"port_xmit_data":138734374764035,"port_rcv_data":138763968967433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833627982039,"port_rcv_data":1192114723987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849195457225,"port_rcv_data":139149747145413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179231514202,"port_rcv_data":1505246225052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.438","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.450","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.463","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.480","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.495","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.512","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.530","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.541","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.3550088,"monotonic":2349961.351336792,"hcas":{"mlx5_0":{"port_xmit_data":138734374764035,"port_rcv_data":138763968967433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833627982039,"port_rcv_data":1192114723987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849195457225,"port_rcv_data":139149747145413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179231514202,"port_rcv_data":1505246225052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:06.455","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":183.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.467","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":209.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.478","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":151.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.489","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":222.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.500","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.509","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.518","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.528","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.3557115,"monotonic":2349962.352040065,"hcas":{"mlx5_0":{"port_xmit_data":138734451349260,"port_rcv_data":138763969043766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833704566999,"port_rcv_data":1192114800387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849195457225,"port_rcv_data":139149747145413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179231514202,"port_rcv_data":1505246225052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.460","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":237.51,"sm_clock_mhz":2422.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.476","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":224.35,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.494","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":241.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.512","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":260.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.529","gpu_util_pct":9.0,"memory_used_mib":64563.0,"power_w":200.64,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.539","gpu_util_pct":41.0,"memory_used_mib":64563.0,"power_w":184.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.550","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":194.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.560","gpu_util_pct":17.0,"memory_used_mib":64563.0,"power_w":184.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.3560367,"monotonic":2349963.352363726,"hcas":{"mlx5_0":{"port_xmit_data":138734534592531,"port_rcv_data":138763969135962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833787016527,"port_rcv_data":1192114890860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849369972868,"port_rcv_data":139149747395124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179406040538,"port_rcv_data":1505246486213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":130.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.486","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":84.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.498","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":104.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.508","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":109.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.519","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":201.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.529","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":243.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.539","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":200.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.549","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":235.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.3568385,"monotonic":2349964.353167217,"hcas":{"mlx5_0":{"port_xmit_data":138734534597211,"port_rcv_data":138763969137202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833787021207,"port_rcv_data":1192114892100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849548063366,"port_rcv_data":139149747676242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179584083839,"port_rcv_data":1505246764266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.466","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.478","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.490","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.500","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.510","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":148.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.520","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":161.61,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.529","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":155.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.538","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":156.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.3576648,"monotonic":2349965.353992881,"hcas":{"mlx5_0":{"port_xmit_data":138734534597211,"port_rcv_data":138763969137202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833787021207,"port_rcv_data":1192114892100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849548063871,"port_rcv_data":139149747676800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179584083839,"port_rcv_data":1505246764266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.459","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.470","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.482","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.493","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.503","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":98.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.513","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":87.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.523","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":109.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.532","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":85.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.3584926,"monotonic":2349966.354821722,"hcas":{"mlx5_0":{"port_xmit_data":138734534597211,"port_rcv_data":138763969137202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833787021207,"port_rcv_data":1192114892100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849548063871,"port_rcv_data":139149747676800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179584083839,"port_rcv_data":1505246764266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.474","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.487","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.498","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.509","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.519","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.529","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.539","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.548","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.359298,"monotonic":2349967.35562719,"hcas":{"mlx5_0":{"port_xmit_data":138734534597211,"port_rcv_data":138763969137202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833787021207,"port_rcv_data":1192114892100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849548063871,"port_rcv_data":139149747676800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179584083839,"port_rcv_data":1505246764266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.474","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.485","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.496","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.506","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.516","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.525","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.534","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.359701,"monotonic":2349968.35602991,"hcas":{"mlx5_0":{"port_xmit_data":138734534597476,"port_rcv_data":138763969137415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833787021207,"port_rcv_data":1192114892100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849548063871,"port_rcv_data":139149747676800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179584083839,"port_rcv_data":1505246764266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:13.459","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.489","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.506","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.524","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.540","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.551","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.562","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.360491,"monotonic":2349969.356820105,"hcas":{"mlx5_0":{"port_xmit_data":138734617840617,"port_rcv_data":138763969230473,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1833869470735,"port_rcv_data":1192114984139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849548068892,"port_rcv_data":139149747678305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179584087347,"port_rcv_data":1505246765382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.477","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":185.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.490","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":182.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.502","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":172.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.514","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":166.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.525","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.534","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.543","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.553","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.3613117,"monotonic":2349970.357638943,"hcas":{"mlx5_0":{"port_xmit_data":138734771015217,"port_rcv_data":138763969384215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834022645335,"port_rcv_data":1192115137875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138849842175239,"port_rcv_data":139149748099935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2179878346203,"port_rcv_data":1505247177799,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.484","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":322.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.499","gpu_util_pct":94.0,"memory_used_mib":59349.0,"power_w":322.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.514","gpu_util_pct":94.0,"memory_used_mib":59349.0,"power_w":349.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.528","gpu_util_pct":96.0,"memory_used_mib":59349.0,"power_w":301.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.542","gpu_util_pct":90.0,"memory_used_mib":64563.0,"power_w":236.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.563","gpu_util_pct":58.0,"memory_used_mib":64563.0,"power_w":286.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.578","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":289.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.591","gpu_util_pct":82.0,"memory_used_mib":64563.0,"power_w":277.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.3620634,"monotonic":2349971.35839144,"hcas":{"mlx5_0":{"port_xmit_data":138734771015217,"port_rcv_data":138763969384215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834022645335,"port_rcv_data":1192115137875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850016881845,"port_rcv_data":139149748381716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180052903288,"port_rcv_data":1505247473325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:16.467","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":148.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.480","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":151.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.492","gpu_util_pct":3.0,"memory_used_mib":59349.0,"power_w":177.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.503","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.513","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":331.96,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.523","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":333.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.532","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":326.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.542","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":324.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.3628702,"monotonic":2349972.359198848,"hcas":{"mlx5_0":{"port_xmit_data":138734771015217,"port_rcv_data":138763969384215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834022645335,"port_rcv_data":1192115137875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850037020405,"port_rcv_data":139149748434279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180073030742,"port_rcv_data":1505247525452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.484","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.495","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.506","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.517","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":103.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.527","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":103.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.536","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":105.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.546","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":88.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.3637135,"monotonic":2349973.360042517,"hcas":{"mlx5_0":{"port_xmit_data":138734771015217,"port_rcv_data":138763969384215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834022645335,"port_rcv_data":1192115137875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850037020405,"port_rcv_data":139149748434279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180073030742,"port_rcv_data":1505247525452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.444","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.456","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.478","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.489","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.499","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.508","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.518","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.3645456,"monotonic":2349974.360874596,"hcas":{"mlx5_0":{"port_xmit_data":138734771015217,"port_rcv_data":138763969384215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834022645335,"port_rcv_data":1192115137875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850037020405,"port_rcv_data":139149748434279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180073030742,"port_rcv_data":1505247525452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.477","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.489","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.501","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.512","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.522","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.533","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.543","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.553","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.3653417,"monotonic":2349975.361671282,"hcas":{"mlx5_0":{"port_xmit_data":138734771015612,"port_rcv_data":138763969384558,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834022645335,"port_rcv_data":1192115137875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850037020405,"port_rcv_data":139149748434279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180073030742,"port_rcv_data":1505247525452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.467","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.479","gpu_util_pct":10.0,"memory_used_mib":59349.0,"power_w":80.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.491","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.502","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.512","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.523","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.534","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.543","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.3661528,"monotonic":2349976.362481847,"hcas":{"mlx5_0":{"port_xmit_data":138734771015612,"port_rcv_data":138763969384558,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834022645335,"port_rcv_data":1192115137875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850037020405,"port_rcv_data":139149748434279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180073030742,"port_rcv_data":1505247525452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:21.481","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.493","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.504","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.515","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.525","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.68,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.544","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.553","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.3669624,"monotonic":2349977.363291084,"hcas":{"mlx5_0":{"port_xmit_data":138734805201945,"port_rcv_data":138763969430327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834056037335,"port_rcv_data":1192115182174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850083144909,"port_rcv_data":139149748490172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180119152722,"port_rcv_data":1505247580601,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.448","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":95.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.460","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.472","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.482","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.492","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.502","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.511","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.521","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.3678224,"monotonic":2349978.364149487,"hcas":{"mlx5_0":{"port_xmit_data":138734881786905,"port_rcv_data":138763969506714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834132622295,"port_rcv_data":1192115258784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850083144909,"port_rcv_data":139149748490172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180119152722,"port_rcv_data":1505247580601,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.473","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":311.82,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.486","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":338.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.498","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":336.78,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.511","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":322.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.524","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":195.1,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.540","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":216.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.553","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":197.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.567","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":228.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.3686147,"monotonic":2349979.364941759,"hcas":{"mlx5_0":{"port_xmit_data":138734881786905,"port_rcv_data":138763969506714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834132622295,"port_rcv_data":1192115258784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850257704559,"port_rcv_data":139149748786156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180293704247,"port_rcv_data":1505247868444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":138.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.480","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":143.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.492","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":177.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.502","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":146.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.513","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":328.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.522","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":299.12,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.532","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":300.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.541","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":305.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.3694577,"monotonic":2349980.36578499,"hcas":{"mlx5_0":{"port_xmit_data":138734881786905,"port_rcv_data":138763969506714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834132622295,"port_rcv_data":1192115258784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850313999260,"port_rcv_data":139149748940785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180349995892,"port_rcv_data":1505248023787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.481","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":106.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.493","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":95.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.504","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":109.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.514","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":104.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.524","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.534","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.543","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":101.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.552","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.3702672,"monotonic":2349981.366596285,"hcas":{"mlx5_0":{"port_xmit_data":138734881786905,"port_rcv_data":138763969506714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834132622295,"port_rcv_data":1192115258784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850313999260,"port_rcv_data":139149748940785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180349995892,"port_rcv_data":1505248023787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.497","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.509","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.521","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.534","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.546","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.559","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.571","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.583","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.371062,"monotonic":2349982.367390158,"hcas":{"mlx5_0":{"port_xmit_data":138734881786905,"port_rcv_data":138763969506714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834132622295,"port_rcv_data":1192115258784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850313999260,"port_rcv_data":139149748940785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180349995892,"port_rcv_data":1505248023787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.537","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.548","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.568","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.578","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.587","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.596","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.3718336,"monotonic":2349983.368162543,"hcas":{"mlx5_0":{"port_xmit_data":138734881787268,"port_rcv_data":138763969507025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834132622295,"port_rcv_data":1192115258784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850313999260,"port_rcv_data":139149748940785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180349995892,"port_rcv_data":1505248023787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.492","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":83.61,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.504","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.515","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.526","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.536","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.546","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.555","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.564","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.3726606,"monotonic":2349984.368989048,"hcas":{"mlx5_0":{"port_xmit_data":138734881787268,"port_rcv_data":138763969507025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834132622295,"port_rcv_data":1192115258784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850313999260,"port_rcv_data":139149748940785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180349995892,"port_rcv_data":1505248023787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.523","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":112.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.534","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.545","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.556","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.566","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.575","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.61,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.3734472,"monotonic":2349985.369774405,"hcas":{"mlx5_0":{"port_xmit_data":138734962043441,"port_rcv_data":138763969597959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834212084855,"port_rcv_data":1192115349738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850429229033,"port_rcv_data":139149749064721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180465223712,"port_rcv_data":1505248145338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.456","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.468","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.480","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.491","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.502","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.512","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.521","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.531","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.3742814,"monotonic":2349986.370609256,"hcas":{"mlx5_0":{"port_xmit_data":138734962043441,"port_rcv_data":138763969597959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834212084855,"port_rcv_data":1192115349738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850429229033,"port_rcv_data":139149749064721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180465223712,"port_rcv_data":1505248145338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.547","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.556","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.565","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.574","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.582","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.590","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.598","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.606","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.3750443,"monotonic":2349987.371373775,"hcas":{"mlx5_0":{"port_xmit_data":138734962043441,"port_rcv_data":138763969597959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834212084855,"port_rcv_data":1192115349738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850429229033,"port_rcv_data":139149749064721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180465223712,"port_rcv_data":1505248145338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.455","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.466","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.477","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.488","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.498","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.507","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.515","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.524","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.3757362,"monotonic":2349988.372063578,"hcas":{"mlx5_0":{"port_xmit_data":138734962043441,"port_rcv_data":138763969597959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834212084855,"port_rcv_data":1192115349738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850429229033,"port_rcv_data":139149749064721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180465223712,"port_rcv_data":1505248145338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.538","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.555","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.576","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.598","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.621","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.642","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.664","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.3764515,"monotonic":2349989.372779169,"hcas":{"mlx5_0":{"port_xmit_data":138734962043441,"port_rcv_data":138763969597959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834212084855,"port_rcv_data":1192115349738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850429229033,"port_rcv_data":139149749064721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180465223712,"port_rcv_data":1505248145338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.500","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.512","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.523","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.534","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.544","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.553","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.563","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.572","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.3772626,"monotonic":2349990.373591123,"hcas":{"mlx5_0":{"port_xmit_data":138734962043441,"port_rcv_data":138763969597959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834212084855,"port_rcv_data":1192115349738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850429229033,"port_rcv_data":139149749064721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180465223712,"port_rcv_data":1505248145338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.513","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.531","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.544","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.554","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.564","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.573","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.582","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.591","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.3780527,"monotonic":2349991.374381107,"hcas":{"mlx5_0":{"port_xmit_data":138734962043441,"port_rcv_data":138763969597959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834212084855,"port_rcv_data":1192115349738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850429229033,"port_rcv_data":139149749064721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180465223712,"port_rcv_data":1505248145338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.524","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":180.87,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.536","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":247.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.549","gpu_util_pct":97.0,"memory_used_mib":59349.0,"power_w":144.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.560","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":160.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.572","gpu_util_pct":60.0,"memory_used_mib":64563.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.582","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":98.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.592","gpu_util_pct":96.0,"memory_used_mib":64563.0,"power_w":97.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.602","gpu_util_pct":1.0,"memory_used_mib":64563.0,"power_w":108.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.3788576,"monotonic":2349992.375185699,"hcas":{"mlx5_0":{"port_xmit_data":138735038628401,"port_rcv_data":138763969673692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834288669815,"port_rcv_data":1192115425703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850603777067,"port_rcv_data":139149749349507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180639774569,"port_rcv_data":1505248433537,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.515","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":212.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.527","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":235.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.538","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":224.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.549","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":245.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.559","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":289.35,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.569","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":298.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.581","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":308.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.591","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":318.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.3796322,"monotonic":2349993.375960568,"hcas":{"mlx5_0":{"port_xmit_data":138735038628401,"port_rcv_data":138763969673692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834288669815,"port_rcv_data":1192115425703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850661577054,"port_rcv_data":139149749515587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180697541696,"port_rcv_data":1505248596820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.462","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":107.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.473","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":88.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.484","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":99.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.495","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":106.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.504","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":204.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.514","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":188.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.523","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":191.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.532","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":207.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.3804696,"monotonic":2349994.376799014,"hcas":{"mlx5_0":{"port_xmit_data":138735038628401,"port_rcv_data":138763969673692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834288669815,"port_rcv_data":1192115425703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850661577054,"port_rcv_data":139149749515587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180697541696,"port_rcv_data":1505248596820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.515","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.527","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.538","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.548","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.558","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.567","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.576","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.585","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":101.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.3812542,"monotonic":2349995.377583348,"hcas":{"mlx5_0":{"port_xmit_data":138735038628401,"port_rcv_data":138763969673692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834288669815,"port_rcv_data":1192115425703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850661577054,"port_rcv_data":139149749515587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180697541696,"port_rcv_data":1505248596820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.483","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.494","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.506","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.516","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.526","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.544","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.553","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.3820946,"monotonic":2349996.378422376,"hcas":{"mlx5_0":{"port_xmit_data":138735038628401,"port_rcv_data":138763969673692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834288669815,"port_rcv_data":1192115425703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850661577054,"port_rcv_data":139149749515587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180697541696,"port_rcv_data":1505248596820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.539","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.552","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.563","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.574","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.602","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.611","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.3828628,"monotonic":2349997.379191013,"hcas":{"mlx5_0":{"port_xmit_data":138735038628666,"port_rcv_data":138763969673905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834288669815,"port_rcv_data":1192115425703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850661577054,"port_rcv_data":139149749515587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180697541696,"port_rcv_data":1505248596820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.485","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.37,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.497","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.508","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.519","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.529","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.539","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.548","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.557","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.3837059,"monotonic":2349998.380033967,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.481","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.493","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.505","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.515","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.526","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.545","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.554","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.3845246,"monotonic":2349999.380853541,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.488","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.500","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.511","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.522","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.533","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.543","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.552","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.561","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.3947015,"monotonic":2350000.391029732,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.506","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.542","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.559","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.574","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.585","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.595","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.605","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.395486,"monotonic":2350001.391814512,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.515","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":85.92,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.528","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.543","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.569","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.590","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.599","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.396278,"monotonic":2350002.392607033,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.500","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":124.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.516","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.533","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.550","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.565","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.576","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.586","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.596","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.397071,"monotonic":2350003.393400453,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.537","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.548","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.559","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.570","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.589","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.598","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.607","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.3978674,"monotonic":2350004.394195218,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.533","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.546","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.568","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.579","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.588","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.597","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.606","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.398659,"monotonic":2350005.394988014,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.548","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.571","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.582","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.602","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.612","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.621","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.399441,"monotonic":2350006.395768705,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.559","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.571","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.582","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.593","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.603","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.612","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.621","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.630","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.4002142,"monotonic":2350007.3965427,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.541","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.554","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.565","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.576","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.586","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:52.596","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:52.605","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:52.614","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.4008255,"monotonic":2350008.397153088,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.543","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.554","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.565","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.574","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.603","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.401626,"monotonic":2350009.397954783,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.540","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.552","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.563","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.574","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.603","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:54.612","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.402416,"monotonic":2350010.398744803,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.527","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.540","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.552","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.564","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.574","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.603","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.4032176,"monotonic":2350011.39954639,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.521","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.533","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.545","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.555","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.565","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.575","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.4037094,"monotonic":2350012.400038572,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.557","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.569","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.580","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.590","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.600","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.611","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.622","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.633","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.4044688,"monotonic":2350013.400798183,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.516","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.533","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.541","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.549","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.556","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.562","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.569","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.4053144,"monotonic":2350014.401642619,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.529","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.539","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.547","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.555","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.563","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.570","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.577","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.4061415,"monotonic":2350015.40247022,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.541","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.550","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.566","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.573","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.586","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.4069648,"monotonic":2350016.403293296,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.531","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.541","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.549","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.557","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.564","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.571","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.578","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.4077907,"monotonic":2350017.404119922,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.541","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.549","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.557","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.566","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.573","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.587","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.4086034,"monotonic":2350018.40493262,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.515","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":132.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.524","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.532","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.540","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.547","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.554","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.561","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.567","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.4094465,"monotonic":2350019.405776029,"hcas":{"mlx5_0":{"port_xmit_data":138735120679814,"port_rcv_data":138763969765906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834369927415,"port_rcv_data":1192115516595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850779499452,"port_rcv_data":139149749642533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180815462076,"port_rcv_data":1505248723021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.524","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":218.01,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.533","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":184.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.542","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":189.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.552","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":182.29,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.560","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.569","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.577","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.584","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.4102786,"monotonic":2350020.406606864,"hcas":{"mlx5_0":{"port_xmit_data":138735273849734,"port_rcv_data":138763969917031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834523097335,"port_rcv_data":1192115667984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138850954057039,"port_rcv_data":139149749936485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2180990026020,"port_rcv_data":1505249023656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.511","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":286.06,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.522","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":340.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.531","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":317.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.541","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":325.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.551","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":243.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.558","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":258.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.565","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":265.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.572","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":256.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.4111166,"monotonic":2350021.407444952,"hcas":{"mlx5_0":{"port_xmit_data":138735273849734,"port_rcv_data":138763969917031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834523097335,"port_rcv_data":1192115667984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851128606406,"port_rcv_data":139149750225565,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181164576150,"port_rcv_data":1505249312894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.568","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":107.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.577","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":104.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.585","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":105.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.593","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":106.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.601","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":300.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.608","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":317.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.614","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":310.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.621","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":313.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.4119062,"monotonic":2350022.408235017,"hcas":{"mlx5_0":{"port_xmit_data":138735273849734,"port_rcv_data":138763969917031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834523097335,"port_rcv_data":1192115667984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851153137654,"port_rcv_data":139149750289078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181189107590,"port_rcv_data":1505249379701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.508","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.516","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.533","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.540","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.548","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.555","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:07.562","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.4127607,"monotonic":2350023.409089297,"hcas":{"mlx5_0":{"port_xmit_data":138735273849734,"port_rcv_data":138763969917031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834523097335,"port_rcv_data":1192115667984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851153137654,"port_rcv_data":139149750289078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181189107590,"port_rcv_data":1505249379701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.490","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.500","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.02,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.509","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.516","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.524","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.533","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.547","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.565","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.4135847,"monotonic":2350024.409913933,"hcas":{"mlx5_0":{"port_xmit_data":138735273849734,"port_rcv_data":138763969917031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834523097335,"port_rcv_data":1192115667984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851153137654,"port_rcv_data":139149750289078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181189107590,"port_rcv_data":1505249379701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.492","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.502","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.510","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.518","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.525","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.532","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.539","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.546","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.414454,"monotonic":2350025.41078284,"hcas":{"mlx5_0":{"port_xmit_data":138735273850389,"port_rcv_data":138763969917634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834523097335,"port_rcv_data":1192115667984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851153137654,"port_rcv_data":139149750289078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181189107590,"port_rcv_data":1505249379701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.514","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.526","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.535","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.545","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.554","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.562","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.571","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.415276,"monotonic":2350026.411605307,"hcas":{"mlx5_0":{"port_xmit_data":138735273850389,"port_rcv_data":138763969917634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834523097335,"port_rcv_data":1192115667984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851153137654,"port_rcv_data":139149750289078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181189107590,"port_rcv_data":1505249379701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.535","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.545","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.47,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.554","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.563","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.572","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.581","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.589","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.597","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.415713,"monotonic":2350027.412041822,"hcas":{"mlx5_0":{"port_xmit_data":138735313421553,"port_rcv_data":138763969968067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834561874199,"port_rcv_data":1192115716468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851207339487,"port_rcv_data":139149750353219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181243306866,"port_rcv_data":1505249442569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.538","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":138.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.551","gpu_util_pct":14.0,"memory_used_mib":59349.0,"power_w":106.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.561","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":137.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.571","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":98.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.589","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.598","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.607","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.4165108,"monotonic":2350028.41283952,"hcas":{"mlx5_0":{"port_xmit_data":138735390006513,"port_rcv_data":138763970045257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834638459159,"port_rcv_data":1192115793720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851207339487,"port_rcv_data":139149750353219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181243306866,"port_rcv_data":1505249442569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.511","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":352.25,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.521","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":339.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.531","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":342.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.542","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":363.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.551","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":211.76,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.558","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":224.05,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.565","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":238.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.573","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":216.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.4173489,"monotonic":2350029.413677838,"hcas":{"mlx5_0":{"port_xmit_data":138735390006513,"port_rcv_data":138763970045257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834638459159,"port_rcv_data":1192115793720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851439907905,"port_rcv_data":139149750792324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181475859863,"port_rcv_data":1505249880151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.514","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":130.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.524","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":105.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.534","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":108.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.543","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":133.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.553","gpu_util_pct":46.0,"memory_used_mib":64563.0,"power_w":287.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.561","gpu_util_pct":77.0,"memory_used_mib":64563.0,"power_w":260.33,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.570","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":294.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.579","gpu_util_pct":54.0,"memory_used_mib":64563.0,"power_w":257.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.4181795,"monotonic":2350030.414508895,"hcas":{"mlx5_0":{"port_xmit_data":138735390006513,"port_rcv_data":138763970045257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834638459159,"port_rcv_data":1192115793720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851439907905,"port_rcv_data":139149750792324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181475859863,"port_rcv_data":1505249880151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.508","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":104.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.517","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.527","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.536","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":109.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.544","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.553","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":101.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.562","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":101.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.570","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":110.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.419019,"monotonic":2350031.415348435,"hcas":{"mlx5_0":{"port_xmit_data":138735390006513,"port_rcv_data":138763970045257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834638459159,"port_rcv_data":1192115793720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851439907905,"port_rcv_data":139149750792324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181475859863,"port_rcv_data":1505249880151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.517","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.526","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.536","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.545","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.554","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.563","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.572","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.581","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.419851,"monotonic":2350032.416180337,"hcas":{"mlx5_0":{"port_xmit_data":138735390006513,"port_rcv_data":138763970045257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834638459159,"port_rcv_data":1192115793720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851439907905,"port_rcv_data":139149750792324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181475859863,"port_rcv_data":1505249880151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.539","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":100.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.550","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.571","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.582","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.605","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.616","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.4206316,"monotonic":2350033.416961155,"hcas":{"mlx5_0":{"port_xmit_data":138735390006778,"port_rcv_data":138763970045470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834638459159,"port_rcv_data":1192115793720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851439907905,"port_rcv_data":139149750792324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181475859863,"port_rcv_data":1505249880151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.499","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.509","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.518","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.527","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.535","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.543","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.551","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.558","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.4214916,"monotonic":2350034.4178213,"hcas":{"mlx5_0":{"port_xmit_data":138735390006778,"port_rcv_data":138763970045470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834638459159,"port_rcv_data":1192115793720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851439907905,"port_rcv_data":139149750792324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181475859863,"port_rcv_data":1505249880151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.532","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.542","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.552","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.561","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.571","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.579","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.588","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.597","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.4223146,"monotonic":2350035.418643033,"hcas":{"mlx5_0":{"port_xmit_data":138735549241417,"port_rcv_data":138763970214560,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834796898915,"port_rcv_data":1192115963739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851558727663,"port_rcv_data":139149750920974,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181594678807,"port_rcv_data":1505250008072,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.528","gpu_util_pct":99.0,"memory_used_mib":59349.0,"power_w":271.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.537","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":265.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.546","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":277.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.555","gpu_util_pct":99.0,"memory_used_mib":59349.0,"power_w":250.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.564","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":130.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.574","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":148.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.582","gpu_util_pct":91.0,"memory_used_mib":64563.0,"power_w":141.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.589","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":147.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.4231467,"monotonic":2350036.419475648,"hcas":{"mlx5_0":{"port_xmit_data":138735625826377,"port_rcv_data":138763970290092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834873483875,"port_rcv_data":1192116039333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851733289523,"port_rcv_data":139149751219695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181769240829,"port_rcv_data":1505250306816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.516","gpu_util_pct":22.0,"memory_used_mib":59349.0,"power_w":263.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":265.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.533","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":255.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.541","gpu_util_pct":4.0,"memory_used_mib":59349.0,"power_w":286.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.549","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":303.86,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.557","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":308.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.564","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":313.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:21.571","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":301.05,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.4237015,"monotonic":2350037.420030563,"hcas":{"mlx5_0":{"port_xmit_data":138735625826377,"port_rcv_data":138763970290092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834873483875,"port_rcv_data":1192116039333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138851907836338,"port_rcv_data":139149751505030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2181943782170,"port_rcv_data":1505250588459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.503","gpu_util_pct":4.0,"memory_used_mib":59349.0,"power_w":229.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.513","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":201.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.522","gpu_util_pct":12.0,"memory_used_mib":59349.0,"power_w":227.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.532","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":205.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.541","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":258.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.549","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":249.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.557","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":251.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.565","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":276.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.4245627,"monotonic":2350038.420891287,"hcas":{"mlx5_0":{"port_xmit_data":138735702411337,"port_rcv_data":138763970367220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834950068835,"port_rcv_data":1192116116693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852106443901,"port_rcv_data":139149751846042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182142403873,"port_rcv_data":1505250946000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.547","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":280.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.558","gpu_util_pct":98.0,"memory_used_mib":59349.0,"power_w":270.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.568","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":280.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.577","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":294.72,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.587","gpu_util_pct":66.0,"memory_used_mib":64563.0,"power_w":231.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:23.597","gpu_util_pct":34.0,"memory_used_mib":64563.0,"power_w":241.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:23.605","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":278.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:23.613","gpu_util_pct":58.0,"memory_used_mib":64563.0,"power_w":253.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.4253652,"monotonic":2350039.42169454,"hcas":{"mlx5_0":{"port_xmit_data":138735816092169,"port_rcv_data":138763970477440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834989557923,"port_rcv_data":1192116156116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852164765623,"port_rcv_data":139149752001312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182200704888,"port_rcv_data":1505251111805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.527","gpu_util_pct":97.0,"memory_used_mib":59349.0,"power_w":246.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.536","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":259.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.546","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":248.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.555","gpu_util_pct":98.0,"memory_used_mib":59349.0,"power_w":238.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.564","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":321.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.572","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":298.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.581","gpu_util_pct":98.0,"memory_used_mib":64563.0,"power_w":302.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.588","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":331.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.4261978,"monotonic":2350040.42252686,"hcas":{"mlx5_0":{"port_xmit_data":138735816092169,"port_rcv_data":138763970477440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834989557923,"port_rcv_data":1192116156116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852339306308,"port_rcv_data":139149752278904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182375246138,"port_rcv_data":1505251390390,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.526","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":195.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.535","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":178.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.543","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":172.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.552","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":201.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:25.560","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":246.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:25.568","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":260.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:25.576","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":249.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:25.584","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":249.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.42704,"monotonic":2350041.423368716,"hcas":{"mlx5_0":{"port_xmit_data":138735816092434,"port_rcv_data":138763970477653,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834989557923,"port_rcv_data":1192116156116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852393206684,"port_rcv_data":139149752430179,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182429138229,"port_rcv_data":1505251534964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.529","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.539","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.548","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.557","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.565","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":175.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.573","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":163.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":200.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.587","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":178.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.4276931,"monotonic":2350042.424022715,"hcas":{"mlx5_0":{"port_xmit_data":138735816092829,"port_rcv_data":138763970477996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1834989557923,"port_rcv_data":1192116156116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852393206832,"port_rcv_data":139149752430379,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182429138229,"port_rcv_data":1505251534964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.507","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.517","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.527","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.536","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.545","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.554","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.563","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.572","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.4285407,"monotonic":2350043.424870092,"hcas":{"mlx5_0":{"port_xmit_data":138735855060776,"port_rcv_data":138763970526587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835027730695,"port_rcv_data":1192116202492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852393212114,"port_rcv_data":139149752432209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182429142845,"port_rcv_data":1505251536204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.534","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":204.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.546","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":218.0,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.558","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":222.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.568","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":210.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.578","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.588","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.597","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.607","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.4293554,"monotonic":2350044.425684153,"hcas":{"mlx5_0":{"port_xmit_data":138735931651855,"port_rcv_data":138763970604384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835104321507,"port_rcv_data":1192116279744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852446509382,"port_rcv_data":139149752493651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182482439985,"port_rcv_data":1505251597723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.521","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":239.69,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.531","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":220.99,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.540","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":233.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.550","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":236.57,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.559","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":199.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.566","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":229.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.573","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":213.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.581","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":230.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.4302173,"monotonic":2350045.426545376,"hcas":{"mlx5_0":{"port_xmit_data":138735992159126,"port_rcv_data":138763970676492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835209507291,"port_rcv_data":1192116394816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852621032485,"port_rcv_data":139149752751386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182656937883,"port_rcv_data":1505251831144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.534","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":155.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.546","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":124.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.557","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":110.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.568","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":114.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.578","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":195.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.588","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":207.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.598","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":185.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.608","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":212.11,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.4310453,"monotonic":2350046.427373059,"hcas":{"mlx5_0":{"port_xmit_data":138735992164461,"port_rcv_data":138763970678335,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835209511971,"port_rcv_data":1192116396056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852798015652,"port_rcv_data":139149753053491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182833886679,"port_rcv_data":1505252123041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.525","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.537","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.549","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.570","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":190.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.580","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":169.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.590","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":177.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.599","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":156.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.4317234,"monotonic":2350047.428051775,"hcas":{"mlx5_0":{"port_xmit_data":138736069425750,"port_rcv_data":138763970765090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835285977824,"port_rcv_data":1192116483417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852820746059,"port_rcv_data":139149753080122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182861368287,"port_rcv_data":1505252153623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.532","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":241.9,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.544","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":222.84,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.556","gpu_util_pct":97.0,"memory_used_mib":59349.0,"power_w":215.7,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.566","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":219.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.576","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.585","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.594","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.603","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.4325473,"monotonic":2350048.428875877,"hcas":{"mlx5_0":{"port_xmit_data":138736146015572,"port_rcv_data":138763970842035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835362568035,"port_rcv_data":1192116560036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138852908757770,"port_rcv_data":139149753175325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2182944627155,"port_rcv_data":1505252244084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.518","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":234.55,"sm_clock_mhz":2422.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.526","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":245.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.534","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":236.56,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.543","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":234.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.550","gpu_util_pct":24.0,"memory_used_mib":64563.0,"power_w":218.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.558","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":212.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.565","gpu_util_pct":33.0,"memory_used_mib":64563.0,"power_w":209.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:33.571","gpu_util_pct":17.0,"memory_used_mib":64563.0,"power_w":222.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.4334052,"monotonic":2350049.429734161,"hcas":{"mlx5_0":{"port_xmit_data":138736146015572,"port_rcv_data":138763970842035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835362568035,"port_rcv_data":1192116560036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853083321352,"port_rcv_data":139149753476171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183119184929,"port_rcv_data":1505252539277,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.531","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":84.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.540","gpu_util_pct":21.0,"memory_used_mib":59349.0,"power_w":85.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.549","gpu_util_pct":99.0,"memory_used_mib":59349.0,"power_w":96.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":86.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.569","gpu_util_pct":96.0,"memory_used_mib":64563.0,"power_w":252.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.576","gpu_util_pct":96.0,"memory_used_mib":64563.0,"power_w":253.96,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.583","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":301.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.589","gpu_util_pct":96.0,"memory_used_mib":64563.0,"power_w":262.79,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.4342542,"monotonic":2350050.430582631,"hcas":{"mlx5_0":{"port_xmit_data":138736146015787,"port_rcv_data":138763970842211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835362568035,"port_rcv_data":1192116560036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853142575243,"port_rcv_data":139149753633640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183178404626,"port_rcv_data":1505252694615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.530","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":221.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.540","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":216.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.549","gpu_util_pct":45.0,"memory_used_mib":59349.0,"power_w":223.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":199.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.570","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.578","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.586","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":85.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.593","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":103.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.4350893,"monotonic":2350051.431418823,"hcas":{"mlx5_0":{"port_xmit_data":138736222600894,"port_rcv_data":138763970917864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835439152995,"port_rcv_data":1192116636177,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853142575243,"port_rcv_data":139149753633640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183178404626,"port_rcv_data":1505252694615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.533","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":245.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.541","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":232.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.550","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":239.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":257.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.565","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":221.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.572","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":194.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.579","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":218.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.586","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":210.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.4357119,"monotonic":2350052.432040084,"hcas":{"mlx5_0":{"port_xmit_data":138736304051265,"port_rcv_data":138763971009945,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835519808782,"port_rcv_data":1192116726619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853371816568,"port_rcv_data":139149753940902,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183414931563,"port_rcv_data":1505253014026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.537","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":117.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.546","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.554","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.563","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.570","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":280.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.577","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":273.33,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.585","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":291.1,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.591","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":289.84,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.4365542,"monotonic":2350053.432883511,"hcas":{"mlx5_0":{"port_xmit_data":138736304054911,"port_rcv_data":138763971010782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835519812355,"port_rcv_data":1192116727441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853493304991,"port_rcv_data":139149754141921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183529123939,"port_rcv_data":1505253215312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.569","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.578","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.586","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.594","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.601","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.608","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.615","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.622","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":86.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.437368,"monotonic":2350054.433697372,"hcas":{"mlx5_0":{"port_xmit_data":138736304055306,"port_rcv_data":138763971011125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835519812355,"port_rcv_data":1192116727441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853493304991,"port_rcv_data":139149754141921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183529123939,"port_rcv_data":1505253215312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.549","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":138.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.566","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.574","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.581","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.588","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.595","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:39.602","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.4382,"monotonic":2350055.434529485,"hcas":{"mlx5_0":{"port_xmit_data":138736304055571,"port_rcv_data":138763971011338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835519812355,"port_rcv_data":1192116727441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853493304991,"port_rcv_data":139149754141921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183529123939,"port_rcv_data":1505253215312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.553","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.562","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.570","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.579","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.586","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.594","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.601","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.607","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.4390364,"monotonic":2350056.435365303,"hcas":{"mlx5_0":{"port_xmit_data":138736388499735,"port_rcv_data":138763971106043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835603463171,"port_rcv_data":1192116820487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853614817116,"port_rcv_data":139149754270629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183650634143,"port_rcv_data":1505253343048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.559","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.568","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.577","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.586","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.594","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.602","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.610","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.617","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.4397101,"monotonic":2350057.436038432,"hcas":{"mlx5_0":{"port_xmit_data":138736388500098,"port_rcv_data":138763971106354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835603463171,"port_rcv_data":1192116820487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853614817116,"port_rcv_data":139149754270629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183650634143,"port_rcv_data":1505253343048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.538","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":138.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.550","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.561","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.572","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.583","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.592","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:42.602","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:42.611","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.4405346,"monotonic":2350058.436863646,"hcas":{"mlx5_0":{"port_xmit_data":138736388500248,"port_rcv_data":138763971106465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835603463171,"port_rcv_data":1192116820487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853614817116,"port_rcv_data":139149754270629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183650634143,"port_rcv_data":1505253343048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.606","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.618","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.629","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.637","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.645","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.655","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.663","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.671","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.441319,"monotonic":2350059.437646936,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.542","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.554","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.566","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.577","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.587","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.597","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.607","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.616","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.4421422,"monotonic":2350060.438470557,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.547","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.559","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.570","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.583","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.602","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:45.623","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:45.644","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:45.664","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.4429061,"monotonic":2350061.439234164,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.546","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.569","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.579","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.589","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.598","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.607","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.616","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.4437025,"monotonic":2350062.4400316,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.546","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.558","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.569","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.580","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.591","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.600","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.609","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.618","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.4445333,"monotonic":2350063.440861426,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.543","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.555","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.567","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.578","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.588","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.597","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.607","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.616","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.7,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.4453828,"monotonic":2350064.441710552,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.599","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.613","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.625","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.636","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.646","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.656","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.666","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.675","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.4461615,"monotonic":2350065.44248899,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.528","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.539","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.550","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.561","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.570","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.579","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:50.588","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:50.597","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.4470134,"monotonic":2350066.443341807,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.587","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.599","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.610","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:51.621","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:51.631","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:51.640","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.649","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.658","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.447807,"monotonic":2350067.444135264,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.548","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.572","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.584","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.594","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.604","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.614","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.623","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.4486437,"monotonic":2350068.444971285,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.565","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.579","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.590","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.600","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.610","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:53.619","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:53.628","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.637","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.449456,"monotonic":2350069.445784612,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.581","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.590","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.598","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.606","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.614","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.621","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.629","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.635","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.4502826,"monotonic":2350070.446610558,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.607","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.620","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.631","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.642","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.652","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.674","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.683","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.4510555,"monotonic":2350071.447383691,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.601","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.614","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.625","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:56.636","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:56.646","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:56.656","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:56.665","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:56.675","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.451713,"monotonic":2350072.448041031,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.600","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.612","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.623","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.634","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.644","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.671","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.452502,"monotonic":2350073.448830044,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.583","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.597","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.608","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.619","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":83.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.629","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:58.639","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.649","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.658","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.453303,"monotonic":2350074.449630988,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.609","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.622","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.635","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.647","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.659","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.670","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.686","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.698","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.45405,"monotonic":2350075.450379225,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.601","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.613","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.624","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:00.635","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:00.645","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:00.655","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:00.664","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:00.673","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.4548392,"monotonic":2350076.451167432,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.572","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.584","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.595","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.606","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.615","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.625","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.634","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.4556568,"monotonic":2350077.451985553,"hcas":{"mlx5_0":{"port_xmit_data":138736472944527,"port_rcv_data":138763971202093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835687113987,"port_rcv_data":1192116914944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.579","gpu_util_pct":53.0,"memory_used_mib":59349.0,"power_w":135.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.592","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.604","gpu_util_pct":46.0,"memory_used_mib":59349.0,"power_w":80.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.617","gpu_util_pct":51.0,"memory_used_mib":59349.0,"power_w":83.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:02.628","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:02.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.648","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.657","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.4564528,"monotonic":2350078.452781469,"hcas":{"mlx5_0":{"port_xmit_data":138736549529487,"port_rcv_data":138763971279345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835763698947,"port_rcv_data":1192116991545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853736329241,"port_rcv_data":139149754399167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183772144347,"port_rcv_data":1505253470520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.578","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":336.71,"sm_clock_mhz":2422.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.591","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":222.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.603","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":350.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.614","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":348.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.626","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":196.91,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.635","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":143.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.644","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":225.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.653","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":146.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.4572558,"monotonic":2350079.453584875,"hcas":{"mlx5_0":{"port_xmit_data":138736549529487,"port_rcv_data":138763971279345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835763698947,"port_rcv_data":1192116991545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853910889135,"port_rcv_data":139149754695892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2183946716120,"port_rcv_data":1505253779293,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.592","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.16,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.603","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":155.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.615","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":140.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.625","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":109.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.635","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":256.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.644","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":318.96,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.653","gpu_util_pct":97.0,"memory_used_mib":64563.0,"power_w":259.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.663","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":305.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.458044,"monotonic":2350080.454373703,"hcas":{"mlx5_0":{"port_xmit_data":138736549529487,"port_rcv_data":138763971279345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835763698947,"port_rcv_data":1192116991545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853969508075,"port_rcv_data":139149754851751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184005310769,"port_rcv_data":1505253929574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.563","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":90.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.574","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":105.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.585","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":96.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.596","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":85.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.605","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":85.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.615","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":99.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.623","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.632","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":93.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.458861,"monotonic":2350081.455190688,"hcas":{"mlx5_0":{"port_xmit_data":138736549529487,"port_rcv_data":138763971279345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835763698947,"port_rcv_data":1192116991545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853969508075,"port_rcv_data":139149754851751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184005310769,"port_rcv_data":1505253929574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.562","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.574","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.584","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.595","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:06.605","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:06.615","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.624","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.633","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.4596882,"monotonic":2350082.456017579,"hcas":{"mlx5_0":{"port_xmit_data":138736549529487,"port_rcv_data":138763971279345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835763698947,"port_rcv_data":1192116991545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853969508075,"port_rcv_data":139149754851751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184005310769,"port_rcv_data":1505253929574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.539","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.550","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.561","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.572","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.582","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.591","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.600","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.609","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.4605482,"monotonic":2350083.456876389,"hcas":{"mlx5_0":{"port_xmit_data":138736549529487,"port_rcv_data":138763971279345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835763698947,"port_rcv_data":1192116991545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853969508075,"port_rcv_data":139149754851751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184005310769,"port_rcv_data":1505253929574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.563","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.576","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.589","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.601","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.612","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.624","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.634","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.649","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.4613688,"monotonic":2350084.45769685,"hcas":{"mlx5_0":{"port_xmit_data":138736549529752,"port_rcv_data":138763971279558,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835763698947,"port_rcv_data":1192116991545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138853969508075,"port_rcv_data":139149754851751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184005310769,"port_rcv_data":1505253929574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.588","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.602","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.613","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.624","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.634","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.643","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259210.462163,"monotonic":2350085.458491346,"hcas":{"mlx5_0":{"port_xmit_data":138736632777703,"port_rcv_data":138763971373304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835846153155,"port_rcv_data":1192117084957,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854089225353,"port_rcv_data":139149754980205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184125026061,"port_rcv_data":1505254058612,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.616","gpu_util_pct":94.0,"memory_used_mib":59349.0,"power_w":237.82,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.628","gpu_util_pct":97.0,"memory_used_mib":59349.0,"power_w":222.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.640","gpu_util_pct":95.0,"memory_used_mib":59349.0,"power_w":233.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.651","gpu_util_pct":94.0,"memory_used_mib":59349.0,"power_w":239.49,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.662","gpu_util_pct":10.0,"memory_used_mib":64563.0,"power_w":109.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.673","gpu_util_pct":75.0,"memory_used_mib":64563.0,"power_w":106.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.685","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":99.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.695","gpu_util_pct":96.0,"memory_used_mib":64563.0,"power_w":111.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.4629164,"monotonic":2350086.459245276,"hcas":{"mlx5_0":{"port_xmit_data":138736709362663,"port_rcv_data":138763971449115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835922738115,"port_rcv_data":1192117160721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854263780421,"port_rcv_data":139149755271622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184299588571,"port_rcv_data":1505254357533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.581","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":219.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.592","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":246.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.603","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":242.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.614","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":255.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.624","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":311.14,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:11.636","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":298.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.647","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":323.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.657","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":319.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.4636986,"monotonic":2350087.460027501,"hcas":{"mlx5_0":{"port_xmit_data":138736709362663,"port_rcv_data":138763971449115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835922738115,"port_rcv_data":1192117160721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854320526265,"port_rcv_data":139149755420697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184356290051,"port_rcv_data":1505254508629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.567","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":98.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.579","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":108.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.590","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":105.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.601","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":107.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.611","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":192.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.620","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":196.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.630","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":164.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.639","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":198.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.4643168,"monotonic":2350088.460644393,"hcas":{"mlx5_0":{"port_xmit_data":138736709362663,"port_rcv_data":138763971449115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835922738115,"port_rcv_data":1192117160721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854320526265,"port_rcv_data":139149755420697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184356290051,"port_rcv_data":1505254508629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.557","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.568","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.580","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.591","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.601","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":105.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:13.611","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":99.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.620","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":93.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.629","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.4651508,"monotonic":2350089.461478502,"hcas":{"mlx5_0":{"port_xmit_data":138736709362663,"port_rcv_data":138763971449115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835922738115,"port_rcv_data":1192117160721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854320526265,"port_rcv_data":139149755420697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184356290051,"port_rcv_data":1505254508629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.564","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.575","gpu_util_pct":84.0,"memory_used_mib":59349.0,"power_w":80.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.586","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":82.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.597","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.606","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.616","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.625","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.633","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.4659753,"monotonic":2350090.462304407,"hcas":{"mlx5_0":{"port_xmit_data":138736709362663,"port_rcv_data":138763971449115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835922738115,"port_rcv_data":1192117160721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854320526265,"port_rcv_data":139149755420697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184356290051,"port_rcv_data":1505254508629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.586","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":157.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.599","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":170.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:15.611","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":195.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:15.623","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":194.24,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:15.633","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.643","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.652","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.661","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.4667974,"monotonic":2350091.463125054,"hcas":{"mlx5_0":{"port_xmit_data":138736785947888,"port_rcv_data":138763971524596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1835999323075,"port_rcv_data":1192117236191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854320526265,"port_rcv_data":139149755420697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184356290051,"port_rcv_data":1505254508629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.572","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":259.9,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.585","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":281.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.597","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":263.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.609","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":254.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.620","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":154.56,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.630","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":167.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.639","gpu_util_pct":39.0,"memory_used_mib":64563.0,"power_w":167.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.649","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":170.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.4676206,"monotonic":2350092.463948217,"hcas":{"mlx5_0":{"port_xmit_data":138736866797749,"port_rcv_data":138763971616042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836079378151,"port_rcv_data":1192117327238,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854495048506,"port_rcv_data":139149755677102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184530810897,"port_rcv_data":1505254763688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.609","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":209.43,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.623","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":255.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.635","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":227.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.646","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":217.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.657","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":196.92,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.668","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":209.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.678","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":212.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.687","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":201.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.4684203,"monotonic":2350093.464747744,"hcas":{"mlx5_0":{"port_xmit_data":138736943387389,"port_rcv_data":138763971693625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836155967791,"port_rcv_data":1192117406026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854669877771,"port_rcv_data":139149755951525,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184705602789,"port_rcv_data":1505255041210,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.626","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":258.98,"sm_clock_mhz":2422.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.640","gpu_util_pct":93.0,"memory_used_mib":59349.0,"power_w":247.27,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.649","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":255.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.659","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":295.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.669","gpu_util_pct":32.0,"memory_used_mib":64563.0,"power_w":268.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.678","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":275.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.687","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":271.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.697","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":289.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.4691775,"monotonic":2350094.465506431,"hcas":{"mlx5_0":{"port_xmit_data":138736943387389,"port_rcv_data":138763971693625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836155967791,"port_rcv_data":1192117406026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854844434643,"port_rcv_data":139149756248540,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184880152942,"port_rcv_data":1505255331175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.596","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.607","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":123.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.618","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":110.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.629","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":109.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.639","gpu_util_pct":41.0,"memory_used_mib":64563.0,"power_w":246.58,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.649","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":258.11,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.658","gpu_util_pct":33.0,"memory_used_mib":64563.0,"power_w":293.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.668","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":246.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.4699793,"monotonic":2350095.466307111,"hcas":{"mlx5_0":{"port_xmit_data":138736943387389,"port_rcv_data":138763971693625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836155967791,"port_rcv_data":1192117406026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854903352851,"port_rcv_data":139149756405060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184939072791,"port_rcv_data":1505255493402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.571","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.586","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.606","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":83.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.630","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":83.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:20.651","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.672","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":85.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.683","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.694","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.4707634,"monotonic":2350096.467091312,"hcas":{"mlx5_0":{"port_xmit_data":138736943387389,"port_rcv_data":138763971693625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836155967791,"port_rcv_data":1192117406026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854903352851,"port_rcv_data":139149756405060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184939072791,"port_rcv_data":1505255493402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.593","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.606","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.617","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.628","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":83.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.648","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.657","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.667","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.4715593,"monotonic":2350097.46788835,"hcas":{"mlx5_0":{"port_xmit_data":138736943387654,"port_rcv_data":138763971693838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836155967791,"port_rcv_data":1192117406026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854903352851,"port_rcv_data":139149756405060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184939072791,"port_rcv_data":1505255493402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.576","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.589","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.601","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.612","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:22.622","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.632","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.641","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.651","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.4723797,"monotonic":2350098.468708418,"hcas":{"mlx5_0":{"port_xmit_data":138737026630892,"port_rcv_data":138763971787784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836238417319,"port_rcv_data":1192117496438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138854903357807,"port_rcv_data":139149756406500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2184939076299,"port_rcv_data":1505255494518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.571","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.583","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.595","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.606","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.618","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.628","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.648","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.4731996,"monotonic":2350099.469528544,"hcas":{"mlx5_0":{"port_xmit_data":138737026636162,"port_rcv_data":138763971789562,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836238421999,"port_rcv_data":1192117497678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855023070129,"port_rcv_data":139149756534794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185058788083,"port_rcv_data":1505255620704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.588","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.601","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.612","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.622","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.633","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.644","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.663","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.4740236,"monotonic":2350100.470350867,"hcas":{"mlx5_0":{"port_xmit_data":138737110483493,"port_rcv_data":138763971884484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836321474575,"port_rcv_data":1192117591251,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855143683625,"port_rcv_data":139149756663918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185179400767,"port_rcv_data":1505255749354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.580","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.592","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.603","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:25.614","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:25.624","gpu_util_pct":1.0,"memory_used_mib":64563.0,"power_w":79.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:25.634","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.643","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.31,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.4748378,"monotonic":2350101.471166085,"hcas":{"mlx5_0":{"port_xmit_data":138737187068453,"port_rcv_data":138763971960946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836398059535,"port_rcv_data":1192117668178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855143683625,"port_rcv_data":139149756663918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185179400767,"port_rcv_data":1505255749354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.580","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":320.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.595","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":285.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.607","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":322.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.617","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":289.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.628","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":200.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.638","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":185.05,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.648","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":211.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.658","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":164.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.4756358,"monotonic":2350102.471965157,"hcas":{"mlx5_0":{"port_xmit_data":138737187068453,"port_rcv_data":138763971960946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836398059535,"port_rcv_data":1192117668178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855318232550,"port_rcv_data":139149756949100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185353945727,"port_rcv_data":1505256030648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.573","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":179.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:27.586","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":183.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:27.597","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":171.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:27.607","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":155.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.617","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":323.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.627","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":307.92,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.636","gpu_util_pct":96.0,"memory_used_mib":64563.0,"power_w":307.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.646","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":320.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.476151,"monotonic":2350103.472478383,"hcas":{"mlx5_0":{"port_xmit_data":138737187068453,"port_rcv_data":138763971960946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836398059535,"port_rcv_data":1192117668178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855375589724,"port_rcv_data":139149757103365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185411303898,"port_rcv_data":1505256194145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.579","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.593","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.604","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.615","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.625","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":92.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.635","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.644","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":88.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.654","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":123.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.4768953,"monotonic":2350104.473223879,"hcas":{"mlx5_0":{"port_xmit_data":138737187068453,"port_rcv_data":138763971960946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836398059535,"port_rcv_data":1192117668178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855375589724,"port_rcv_data":139149757103365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185411303898,"port_rcv_data":1505256194145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.560","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.573","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.584","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.595","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.604","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.614","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.623","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.632","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.477734,"monotonic":2350105.474063147,"hcas":{"mlx5_0":{"port_xmit_data":138737187068453,"port_rcv_data":138763971960946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836398059535,"port_rcv_data":1192117668178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855375589724,"port_rcv_data":139149757103365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185411303898,"port_rcv_data":1505256194145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.583","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.595","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.606","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.617","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.627","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.636","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.646","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.655","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.478558,"monotonic":2350106.474887109,"hcas":{"mlx5_0":{"port_xmit_data":138737187068453,"port_rcv_data":138763971960946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836398059535,"port_rcv_data":1192117668178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855375589724,"port_rcv_data":139149757103365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185411303898,"port_rcv_data":1505256194145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.584","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.597","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.608","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.619","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.629","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.647","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.656","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.4793768,"monotonic":2350107.475706066,"hcas":{"mlx5_0":{"port_xmit_data":138737187068718,"port_rcv_data":138763971961159,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836398059535,"port_rcv_data":1192117668178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855375589724,"port_rcv_data":139149757103365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185411303898,"port_rcv_data":1505256194145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.561","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.573","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.584","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.595","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.605","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.616","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.626","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.636","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.4797225,"monotonic":2350108.476050293,"hcas":{"mlx5_0":{"port_xmit_data":138737269120028,"port_rcv_data":138763972053245,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836479317135,"port_rcv_data":1192117758667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855493512057,"port_rcv_data":139149757229564,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185529224278,"port_rcv_data":1505256321105,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.603","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.616","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.628","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:33.641","gpu_util_pct":8.0,"memory_used_mib":59349.0,"power_w":81.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.671","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.680","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.4805121,"monotonic":2350109.476841193,"hcas":{"mlx5_0":{"port_xmit_data":138737345704988,"port_rcv_data":138763972128823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836555902095,"port_rcv_data":1192117834586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855493512057,"port_rcv_data":139149757229564,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185529224278,"port_rcv_data":1505256321105,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.619","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":314.04,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.632","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":306.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.645","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":358.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.657","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":355.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.669","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":229.7,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.678","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":217.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.688","gpu_util_pct":76.0,"memory_used_mib":64563.0,"power_w":226.05,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.697","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":236.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.4812913,"monotonic":2350110.477620065,"hcas":{"mlx5_0":{"port_xmit_data":138737345704988,"port_rcv_data":138763972128823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836555902095,"port_rcv_data":1192117834586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855668077655,"port_rcv_data":139149757531821,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185703776715,"port_rcv_data":1505256610915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.624","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":153.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.638","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":103.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:35.650","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":134.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:35.661","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":137.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.671","gpu_util_pct":99.0,"memory_used_mib":64563.0,"power_w":278.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.681","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":269.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.690","gpu_util_pct":28.0,"memory_used_mib":64563.0,"power_w":280.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.700","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":263.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.4820614,"monotonic":2350111.478389683,"hcas":{"mlx5_0":{"port_xmit_data":138737345704988,"port_rcv_data":138763972128823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836555902095,"port_rcv_data":1192117834586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855727166786,"port_rcv_data":139149757692453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185762848065,"port_rcv_data":1505256771965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.564","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.23,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.578","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":101.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.590","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":110.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.601","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":91.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.611","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":98.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.621","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":104.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.631","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":102.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.640","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":109.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.4829383,"monotonic":2350112.479266087,"hcas":{"mlx5_0":{"port_xmit_data":138737345704988,"port_rcv_data":138763972128823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836555902095,"port_rcv_data":1192117834586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855727166786,"port_rcv_data":139149757692453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185762848065,"port_rcv_data":1505256771965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.571","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.591","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:37.603","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:37.613","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.623","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.633","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.642","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.651","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.4837055,"monotonic":2350113.480034325,"hcas":{"mlx5_0":{"port_xmit_data":138737345704988,"port_rcv_data":138763972128823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836555902095,"port_rcv_data":1192117834586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855727166786,"port_rcv_data":139149757692453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185762848065,"port_rcv_data":1505256771965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.585","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":97.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.597","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":84.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.608","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":83.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.619","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":84.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.629","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.647","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.656","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.4845295,"monotonic":2350114.480858493,"hcas":{"mlx5_0":{"port_xmit_data":138737345704988,"port_rcv_data":138763972128823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836555902095,"port_rcv_data":1192117834586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855727166786,"port_rcv_data":139149757692453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185762848065,"port_rcv_data":1505256771965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.588","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":228.45,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.600","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":223.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.613","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":218.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.625","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":200.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.637","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.650","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.661","gpu_util_pct":31.0,"memory_used_mib":64563.0,"power_w":81.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.677","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.485327,"monotonic":2350115.481655682,"hcas":{"mlx5_0":{"port_xmit_data":138737422290213,"port_rcv_data":138763972204536,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836632487055,"port_rcv_data":1192117910427,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855727166786,"port_rcv_data":139149757692453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185762848065,"port_rcv_data":1505256771965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.608","gpu_util_pct":11.0,"memory_used_mib":59349.0,"power_w":254.49,"sm_clock_mhz":2422.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.620","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":260.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.632","gpu_util_pct":19.0,"memory_used_mib":59349.0,"power_w":255.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:40.643","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":264.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:40.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":215.04,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":212.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.672","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":240.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.681","gpu_util_pct":19.0,"memory_used_mib":64563.0,"power_w":209.92,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.4861326,"monotonic":2350116.482461055,"hcas":{"mlx5_0":{"port_xmit_data":138737506131820,"port_rcv_data":138763972297444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836715533779,"port_rcv_data":1192118002296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138855901715340,"port_rcv_data":139149757974907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2185937415105,"port_rcv_data":1505257072506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.614","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.625","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":83.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.637","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.647","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.657","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":250.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.667","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":243.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.678","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":270.43,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.688","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":247.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.4869137,"monotonic":2350117.483243868,"hcas":{"mlx5_0":{"port_xmit_data":138737506137544,"port_rcv_data":138763972298808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836715539631,"port_rcv_data":1192118003660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856080085969,"port_rcv_data":139149758272069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186115768583,"port_rcv_data":1505257370895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.591","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.602","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.614","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.624","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.634","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":120.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.644","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":113.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":104.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.4877381,"monotonic":2350118.484067156,"hcas":{"mlx5_0":{"port_xmit_data":138737506137544,"port_rcv_data":138763972298808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836715539631,"port_rcv_data":1192118003660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856080085969,"port_rcv_data":139149758272069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186115768583,"port_rcv_data":1505257370895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.587","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.599","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.610","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.620","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.630","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":98.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.639","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":105.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.648","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":90.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.657","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":103.05,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.4885647,"monotonic":2350119.484893745,"hcas":{"mlx5_0":{"port_xmit_data":138737506137544,"port_rcv_data":138763972298808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836715539631,"port_rcv_data":1192118003660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856080085969,"port_rcv_data":139149758272069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186115768583,"port_rcv_data":1505257370895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.587","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.599","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":82.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.610","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":83.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.621","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":83.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.631","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.640","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.649","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.658","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.4894028,"monotonic":2350120.48573063,"hcas":{"mlx5_0":{"port_xmit_data":138737506137544,"port_rcv_data":138763972298808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836715539631,"port_rcv_data":1192118003660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856080085969,"port_rcv_data":139149758272069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186115768583,"port_rcv_data":1505257370895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.599","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":160.06,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.616","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":216.25,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.628","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":211.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.640","gpu_util_pct":95.0,"memory_used_mib":59349.0,"power_w":219.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.651","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.660","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.670","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.679","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.4902053,"monotonic":2350121.486534432,"hcas":{"mlx5_0":{"port_xmit_data":138737582722769,"port_rcv_data":138763972374847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836792124591,"port_rcv_data":1192118079892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856080085969,"port_rcv_data":139149758272069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186115768583,"port_rcv_data":1505257370895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.599","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":240.36,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.611","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":253.85,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:46.622","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":237.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.633","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":261.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.643","gpu_util_pct":21.0,"memory_used_mib":64563.0,"power_w":145.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.653","gpu_util_pct":52.0,"memory_used_mib":64563.0,"power_w":149.28,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":159.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.672","gpu_util_pct":29.0,"memory_used_mib":64563.0,"power_w":152.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.4910219,"monotonic":2350122.487351039,"hcas":{"mlx5_0":{"port_xmit_data":138737664769432,"port_rcv_data":138763972467151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836873377511,"port_rcv_data":1192118171466,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856254603919,"port_rcv_data":139149758523963,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186290288289,"port_rcv_data":1505257625767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.588","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.600","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":117.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.611","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":102.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.622","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":100.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.632","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":216.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.642","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":197.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.651","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":237.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.661","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":221.29,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.4918385,"monotonic":2350123.488168334,"hcas":{"mlx5_0":{"port_xmit_data":138737664774112,"port_rcv_data":138763972468391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836873382191,"port_rcv_data":1192118172706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856431931619,"port_rcv_data":139149758798911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186467609570,"port_rcv_data":1505257924725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.576","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.588","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.599","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.609","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":82.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.619","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":177.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.628","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":148.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":145.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.647","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":128.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.492685,"monotonic":2350124.489014584,"hcas":{"mlx5_0":{"port_xmit_data":138737664774112,"port_rcv_data":138763972468391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836873382191,"port_rcv_data":1192118172706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856431932157,"port_rcv_data":139149758799501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186467609570,"port_rcv_data":1505257924725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.611","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.622","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.634","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.645","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.655","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":108.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.664","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.673","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":97.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.682","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":84.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.4934883,"monotonic":2350125.489817752,"hcas":{"mlx5_0":{"port_xmit_data":138737664774112,"port_rcv_data":138763972468391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836873382191,"port_rcv_data":1192118172706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856431932157,"port_rcv_data":139149758799501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186467609570,"port_rcv_data":1505257924725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.574","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.586","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.597","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.608","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.618","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.627","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.636","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.645","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.4943318,"monotonic":2350126.490661303,"hcas":{"mlx5_0":{"port_xmit_data":138737664774112,"port_rcv_data":138763972468391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836873382191,"port_rcv_data":1192118172706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856431932157,"port_rcv_data":139149758799501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186467609570,"port_rcv_data":1505257924725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.622","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.634","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.645","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:51.656","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.667","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.676","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.685","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.694","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.4951262,"monotonic":2350127.491455736,"hcas":{"mlx5_0":{"port_xmit_data":138737664774377,"port_rcv_data":138763972468604,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836873382191,"port_rcv_data":1192118172706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856431932157,"port_rcv_data":139149758799501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186467609570,"port_rcv_data":1505257924725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:52.575","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":92.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:52.587","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:52.598","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.609","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.619","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.629","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.647","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.495971,"monotonic":2350128.492300546,"hcas":{"mlx5_0":{"port_xmit_data":138737749218806,"port_rcv_data":138763972563956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836957033007,"port_rcv_data":1192118266047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856553444477,"port_rcv_data":139149758931489,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186589119774,"port_rcv_data":1505258054724,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.639","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.658","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.670","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.680","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.691","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.700","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.710","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.719","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.4967487,"monotonic":2350129.493077384,"hcas":{"mlx5_0":{"port_xmit_data":138737749218806,"port_rcv_data":138763972563956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1836957033007,"port_rcv_data":1192118266047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856553444477,"port_rcv_data":139149758931489,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186589119774,"port_rcv_data":1505258054724,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.643","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.655","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.666","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.677","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.687","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.697","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.706","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.715","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.11,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.4975266,"monotonic":2350130.493855524,"hcas":{"mlx5_0":{"port_xmit_data":138737825803766,"port_rcv_data":138763972639844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837033617967,"port_rcv_data":1192118342248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856553444477,"port_rcv_data":139149758931489,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186589119774,"port_rcv_data":1505258054724,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.604","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":263.38,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.617","gpu_util_pct":99.0,"memory_used_mib":59349.0,"power_w":215.72,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.629","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":240.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.640","gpu_util_pct":100.0,"memory_used_mib":59349.0,"power_w":237.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.652","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":146.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.667","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":149.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.691","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":153.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.714","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":152.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.4982684,"monotonic":2350131.49459784,"hcas":{"mlx5_0":{"port_xmit_data":138737825803766,"port_rcv_data":138763972639844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837033617967,"port_rcv_data":1192118342248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856727989697,"port_rcv_data":139149759213120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186763683199,"port_rcv_data":1505258354730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.619","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":192.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.631","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":207.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:56.642","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":191.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.653","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":191.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.663","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":324.41,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.676","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":283.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.687","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":322.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.698","gpu_util_pct":100.0,"memory_used_mib":64563.0,"power_w":311.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.4990616,"monotonic":2350132.495390911,"hcas":{"mlx5_0":{"port_xmit_data":138737825803766,"port_rcv_data":138763972639844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837033617967,"port_rcv_data":1192118342248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856786479315,"port_rcv_data":139149759371676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186822156456,"port_rcv_data":1505258511661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.595","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":105.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.607","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":113.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.618","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":102.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.628","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":98.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.638","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":130.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.648","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":133.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.657","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":121.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.666","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":135.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.4998899,"monotonic":2350133.496219373,"hcas":{"mlx5_0":{"port_xmit_data":138737825803766,"port_rcv_data":138763972639844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837033617967,"port_rcv_data":1192118342248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856786479315,"port_rcv_data":139149759371676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186822156456,"port_rcv_data":1505258511661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.600","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":89.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.612","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.623","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.634","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.644","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":102.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.653","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":110.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.662","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":101.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.671","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":114.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.5007107,"monotonic":2350134.497040331,"hcas":{"mlx5_0":{"port_xmit_data":138737825803766,"port_rcv_data":138763972639844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837033617967,"port_rcv_data":1192118342248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856786479315,"port_rcv_data":139149759371676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186822156456,"port_rcv_data":1505258511661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.605","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":122.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.618","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.629","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.639","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.649","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.659","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.8,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.668","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.677","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.501528,"monotonic":2350135.497857481,"hcas":{"mlx5_0":{"port_xmit_data":138737825803766,"port_rcv_data":138763972639844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837033617967,"port_rcv_data":1192118342248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856786479315,"port_rcv_data":139149759371676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186822156456,"port_rcv_data":1505258511661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:00.602","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:00.614","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.625","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.636","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.646","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.656","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.665","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.674","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.5023506,"monotonic":2350136.498679944,"hcas":{"mlx5_0":{"port_xmit_data":138737825803966,"port_rcv_data":138763972639992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837033617967,"port_rcv_data":1192118342248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856786479315,"port_rcv_data":139149759371676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186822156456,"port_rcv_data":1505258511661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.627","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.638","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.650","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.660","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.670","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.680","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.56,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.689","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.698","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.5031524,"monotonic":2350137.4994817,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.604","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.617","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.624","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.632","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.641","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.649","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.657","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.665","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.503696,"monotonic":2350138.500025384,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.603","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.615","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.626","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.637","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.647","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.657","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.666","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.675","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.5045154,"monotonic":2350139.500844806,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.607","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.619","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.630","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.641","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.651","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.660","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.669","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.678","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.83,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.5053403,"monotonic":2350140.501669435,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.652","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.664","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:05.675","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.686","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.696","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.706","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.715","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.724","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.5061197,"monotonic":2350141.502448438,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.627","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.639","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.650","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.661","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.671","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.681","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.690","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.699","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":83.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.5069296,"monotonic":2350142.50325838,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.611","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.623","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.634","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.645","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.654","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.664","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.673","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.682","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.5077574,"monotonic":2350143.504086353,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:08.623","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:08.635","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.646","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.657","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.667","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.677","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.686","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.508574,"monotonic":2350144.504902623,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.626","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.638","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.649","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.660","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.670","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.679","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.688","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.697","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.509383,"monotonic":2350145.505711809,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.655","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.667","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.678","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.691","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.703","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.716","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.728","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.740","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.5101464,"monotonic":2350146.506475149,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.656","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:11.667","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.678","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.689","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.699","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.708","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.717","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.726","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.511013,"monotonic":2350147.50734207,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:12.657","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.669","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.680","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.691","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.701","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.711","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.720","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.730","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.5117111,"monotonic":2350148.508039884,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.632","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.644","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.655","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.666","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.676","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.685","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.704","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.5125155,"monotonic":2350149.508844529,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.657","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.669","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.680","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.691","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.701","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.711","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.720","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":79.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.730","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.513292,"monotonic":2350150.509620916,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.636","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.648","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.659","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.670","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.681","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.690","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.700","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.709","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.514091,"monotonic":2350151.510420127,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:16.664","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:16.676","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:16.687","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.698","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.709","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.718","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.728","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.737","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.514178,"monotonic":2350152.510506453,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.652","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.664","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.675","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.685","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.704","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.713","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.721","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.5149832,"monotonic":2350153.511312588,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:18.651","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:18.663","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.674","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.685","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.704","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.713","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.722","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.5157044,"monotonic":2350154.512032991,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.660","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.672","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.684","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.696","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.710","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.724","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.734","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.744","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.5164804,"monotonic":2350155.512809049,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.653","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.665","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.676","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.688","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.698","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.707","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.717","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.726","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.5172813,"monotonic":2350156.513609821,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.639","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.651","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.662","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.673","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.683","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.693","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.702","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.711","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.518087,"monotonic":2350157.514415763,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:22.645","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:22.657","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.668","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.679","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.690","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.700","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.710","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.720","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.5188816,"monotonic":2350158.515210311,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.624","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.636","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.647","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.657","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.667","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.677","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.686","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.5197008,"monotonic":2350159.516030236,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.641","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.653","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.665","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.676","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.686","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.696","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.706","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.715","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.5205007,"monotonic":2350160.516830099,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.668","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.679","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.690","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.701","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.711","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.720","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.729","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.738","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.5212772,"monotonic":2350161.517605883,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.668","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.680","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.691","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.701","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.711","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.721","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.730","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.739","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":82.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.5220494,"monotonic":2350162.518379115,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:27.634","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.645","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.656","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.667","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.676","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.686","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.703","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.5228603,"monotonic":2350163.519189483,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.678","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.690","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.702","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.716","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.731","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.741","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.751","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.761","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.5236144,"monotonic":2350164.519943628,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.604","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.616","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.627","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.637","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.647","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.657","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.666","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.675","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.5237286,"monotonic":2350165.520055974,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:30.669","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:30.684","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.704","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.726","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.748","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.770","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.781","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.792","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.5244493,"monotonic":2350166.520778017,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.649","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.661","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.672","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.683","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.693","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.702","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.712","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.721","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.5252545,"monotonic":2350167.521582874,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.670","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.682","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.693","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.704","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.716","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.726","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.736","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.745","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.5260274,"monotonic":2350168.522355936,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.648","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.660","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.671","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.682","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.692","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.701","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.711","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.720","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.5268364,"monotonic":2350169.523165072,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.676","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.688","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.699","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.710","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.720","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.729","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.738","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.747","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.5276108,"monotonic":2350170.523940291,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.652","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.664","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.675","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.685","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.705","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.714","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.723","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.5284193,"monotonic":2350171.524748213,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.668","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.680","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.691","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.701","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.712","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.721","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.730","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.739","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.5292013,"monotonic":2350172.525530283,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.666","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.680","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.692","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.705","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.722","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.734","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.745","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.755","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.5299656,"monotonic":2350173.526295001,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.634","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.645","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.657","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.667","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.677","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.686","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.695","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.704","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.5307858,"monotonic":2350174.527115042,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.650","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.662","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.673","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.683","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.693","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.702","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.711","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:39.720","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.5316012,"monotonic":2350175.527929609,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.654","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.667","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.678","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.689","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.698","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.708","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:40.717","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.726","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.5324006,"monotonic":2350176.528728452,"hcas":{"mlx5_0":{"port_xmit_data":138737909052112,"port_rcv_data":138763972733809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837116072175,"port_rcv_data":1192118434395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856906196593,"port_rcv_data":139149759501953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186941871748,"port_rcv_data":1505258640605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.656","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":136.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.667","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.676","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.684","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.693","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.701","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.709","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.717","gpu_util_pct":0.0,"memory_used_mib":64563.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_2.jsonl

@@ -0,0 +1,460 @@
+{"wall_time":1788258842.0985525,"monotonic":2349698.565057843,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:02.219","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:02.235","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.4,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:02.246","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:02.257","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:02.266","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.94,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:02.275","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:02.282","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:02.291","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258843.0993538,"monotonic":2349699.565859306,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:03.244","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:03.256","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.02,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:03.267","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:03.277","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:03.287","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.78,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:03.296","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:03.305","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:03.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258844.1001348,"monotonic":2349700.566640292,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:04.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:04.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:04.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:04.281","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:04.291","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:04.300","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:04.309","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:04.318","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258845.1009078,"monotonic":2349701.567413147,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.225","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.237","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.258","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.268","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.277","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.287","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.296","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.1017087,"monotonic":2349702.568213738,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.9,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.280","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.290","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.299","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.317","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.1024864,"monotonic":2349703.568991462,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.249","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.260","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.66,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.271","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.282","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.291","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.301","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.310","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.319","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.103261,"monotonic":2349704.569766251,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.244","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.255","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.93,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.265","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.276","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.285","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.295","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.305","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.1040437,"monotonic":2349705.57054891,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.252","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.67,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.274","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.284","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.294","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.303","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":77.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.312","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.320","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.1048207,"monotonic":2349706.571326051,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.251","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.265","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":81.15,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.276","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.287","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.296","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.306","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.315","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.324","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.105595,"monotonic":2349707.572098807,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.253","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.265","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":81.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.276","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.287","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.297","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.306","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.316","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.325","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.1063619,"monotonic":2349708.572867213,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.76,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.280","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.289","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.299","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.317","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.11,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.1071408,"monotonic":2349709.573645236,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.280","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.289","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.299","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.316","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.1079218,"monotonic":2349710.574427219,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.249","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.260","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.280","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.290","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.299","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.317","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.108703,"monotonic":2349711.575208491,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.251","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.262","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.273","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.283","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.292","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.302","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.310","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.319","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.109483,"monotonic":2349712.575988161,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.255","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.266","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.277","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.287","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.296","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.305","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.322","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.1102607,"monotonic":2349713.576766217,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.232","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.243","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.254","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.264","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.274","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.283","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.292","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.300","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.1110673,"monotonic":2349714.577570992,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.257","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.268","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.18,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.279","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.289","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.299","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.317","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.326","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.1118486,"monotonic":2349715.578353509,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.252","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":81.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.274","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.284","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.294","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.303","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.311","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.320","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.112552,"monotonic":2349716.579056202,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.279","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.289","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.298","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.307","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.315","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.113344,"monotonic":2349717.579849183,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.231","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.243","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.253","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.273","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.282","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.291","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.300","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.114152,"monotonic":2349718.580657211,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.258","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.281","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.291","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.301","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.310","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.319","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.328","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.1144571,"monotonic":2349719.58096139,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.233","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.244","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.254","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.265","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.274","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.284","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.292","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.301","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.1147437,"monotonic":2349720.581248472,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.237","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.249","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.280","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.289","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.298","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.307","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.1155453,"monotonic":2349721.582050485,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.257","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.268","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.278","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.289","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.299","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.317","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.326","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.1159527,"monotonic":2349722.582457649,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.256","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.267","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.278","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.289","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.298","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.317","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.326","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.1167328,"monotonic":2349723.583238122,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.264","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.275","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.286","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.296","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.306","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.316","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.326","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.335","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.1175203,"monotonic":2349724.584025437,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.265","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.23,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.276","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.286","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.296","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.305","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.323","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.332","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.1183004,"monotonic":2349725.584805519,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.264","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.23,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.274","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.285","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.295","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.304","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.313","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.322","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.330","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.1190863,"monotonic":2349726.585591369,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.260","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.281","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.291","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.301","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.310","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.319","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.328","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.119871,"monotonic":2349727.586376148,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.76,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.281","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.291","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.300","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.309","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.318","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.326","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.1206605,"monotonic":2349728.587165565,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.274","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.285","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.295","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.304","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.323","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.331","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.1214416,"monotonic":2349729.587946883,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.280","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.22,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.290","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.300","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.310","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.319","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.328","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.336","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.1215706,"monotonic":2349730.588075657,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.267","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.278","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.288","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.299","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.317","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.326","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.335","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.1223493,"monotonic":2349731.58885462,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.267","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.278","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":81.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.289","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.299","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.309","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.318","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.327","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.335","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.1231277,"monotonic":2349732.589633179,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.274","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.71,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.285","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.295","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.304","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.313","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.322","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.331","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.1239147,"monotonic":2349733.590419576,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.280","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.291","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.302","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.312","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.322","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.331","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:37.340","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.1246893,"monotonic":2349734.591194721,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.281","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.292","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.302","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.312","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.321","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.1254873,"monotonic":2349735.591992391,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.273","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.285","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.71,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.296","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.306","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.316","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.325","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.335","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.344","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.1262631,"monotonic":2349736.592768128,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.250","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.261","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":81.25,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.271","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.282","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.291","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.301","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.310","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.318","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.1263447,"monotonic":2349737.592849707,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.271","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.282","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.292","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.302","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.311","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.320","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.329","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:41.338","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.127128,"monotonic":2349738.593632815,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.274","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.285","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.295","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.304","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.313","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.321","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.330","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.127929,"monotonic":2349739.594433811,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.277","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.289","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.300","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.310","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.320","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.329","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.338","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.347","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.128701,"monotonic":2349740.595205952,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.272","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.283","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.294","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.304","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.323","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.332","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.341","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.1294827,"monotonic":2349741.595987863,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.277","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.288","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.299","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.308","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.318","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.327","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.336","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.345","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.1297746,"monotonic":2349742.596279737,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.280","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.290","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.300","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.310","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.319","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.328","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.337","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.130559,"monotonic":2349743.597064108,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.231","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.242","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.253","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.272","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.281","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.288","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:47.295","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.13139,"monotonic":2349744.597895156,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.228","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.240","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.250","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.260","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.270","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.279","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.288","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.297","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.1322155,"monotonic":2349745.598720837,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.256","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.266","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.277","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.287","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.297","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.306","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.315","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.324","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.133016,"monotonic":2349746.599521228,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.267","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.279","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.71,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.289","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.299","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.309","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.318","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.327","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.336","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.1335204,"monotonic":2349747.600025542,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.236","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.247","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.51,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.258","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.268","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.277","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.286","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.295","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:51.304","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.1343465,"monotonic":2349748.600851839,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.232","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.243","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.93,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.253","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.263","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.273","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.282","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.292","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.300","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.135175,"monotonic":2349749.60167981,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.232","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.243","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.93,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.254","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.264","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.273","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.283","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.292","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.300","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.1360018,"monotonic":2349750.60250706,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.236","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.17,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.279","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.288","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.297","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.306","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.1368258,"monotonic":2349751.603330697,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.236","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.248","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.259","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.270","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.280","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.289","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.299","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.1376483,"monotonic":2349752.604153556,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.236","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.247","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.258","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.269","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.279","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.289","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.298","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.307","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.1384706,"monotonic":2349753.604975695,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.219","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.230","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.241","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":77.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.251","gpu_util_pct":0.0,"memory_used_mib":57301.0,"power_w":79.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.261","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.270","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.279","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.288","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.1393054,"monotonic":2349754.605810846,"hcas":{"mlx5_0":{"port_xmit_data":294433496178110,"port_rcv_data":294781359854220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163230566,"port_rcv_data":1232311471711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.253","gpu_util_pct":0.0,"memory_used_mib":59573.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.269","gpu_util_pct":0.0,"memory_used_mib":59977.0,"power_w":80.41,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.284","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.297","gpu_util_pct":54.0,"memory_used_mib":62437.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.309","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.319","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.328","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.337","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.140101,"monotonic":2349755.606605954,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.241","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":86.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.253","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":88.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.264","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":91.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.275","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":92.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.285","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.295","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.305","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.140932,"monotonic":2349756.607437037,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.241","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":92.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.254","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":92.62,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.265","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":98.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.276","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.286","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.295","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.305","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.1417546,"monotonic":2349757.608259835,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.223","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.235","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.99,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.246","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":79.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.257","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.267","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.277","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.286","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.295","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.1426017,"monotonic":2349758.609106504,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.245","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.259","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.96,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.271","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.282","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.292","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.302","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.311","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.320","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.1434166,"monotonic":2349759.609921669,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.241","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.253","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.264","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.275","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.285","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.294","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.304","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.313","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.1442404,"monotonic":2349760.610744595,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.250","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.264","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.71,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.275","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.285","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.295","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.305","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.314","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.323","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.1450553,"monotonic":2349761.611560086,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.226","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.238","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":81.15,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.249","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.260","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.270","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.279","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.289","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.298","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.1458957,"monotonic":2349762.612400692,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.226","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.239","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.250","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.261","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.271","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.281","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.290","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.300","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.1467304,"monotonic":2349763.613235759,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.244","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.256","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.267","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.278","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.288","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.297","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.306","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":80.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.316","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.11,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.1475542,"monotonic":2349764.614059408,"hcas":{"mlx5_0":{"port_xmit_data":294433496323065,"port_rcv_data":294781419496195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817163372896,"port_rcv_data":1232371111216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.261","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.273","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.285","gpu_util_pct":0.0,"memory_used_mib":62549.0,"power_w":78.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.297","gpu_util_pct":0.0,"memory_used_mib":62437.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.308","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.318","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.328","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.337","gpu_util_pct":0.0,"memory_used_mib":61689.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.1483703,"monotonic":2349765.614874965,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294526990323693,"port_rcv_data":294749689250948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1882988938478,"port_rcv_data":1193268730252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.254","gpu_util_pct":100.0,"memory_used_mib":65427.0,"power_w":214.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.266","gpu_util_pct":100.0,"memory_used_mib":65427.0,"power_w":184.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.278","gpu_util_pct":100.0,"memory_used_mib":65433.0,"power_w":172.62,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.289","gpu_util_pct":100.0,"memory_used_mib":65427.0,"power_w":290.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.299","gpu_util_pct":99.0,"memory_used_mib":69493.0,"power_w":127.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.310","gpu_util_pct":99.0,"memory_used_mib":69493.0,"power_w":120.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.321","gpu_util_pct":99.0,"memory_used_mib":69493.0,"power_w":122.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.332","gpu_util_pct":100.0,"memory_used_mib":69493.0,"power_w":129.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.149169,"monotonic":2349766.615674017,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.232","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":211.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.244","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":212.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.255","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":222.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.266","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":190.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.276","gpu_util_pct":100.0,"memory_used_mib":70023.0,"power_w":310.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.288","gpu_util_pct":100.0,"memory_used_mib":70023.0,"power_w":287.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.298","gpu_util_pct":100.0,"memory_used_mib":70023.0,"power_w":309.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.308","gpu_util_pct":100.0,"memory_used_mib":70023.0,"power_w":296.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.1495173,"monotonic":2349767.616022623,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.229","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":82.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.241","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":85.02,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.252","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":80.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.263","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":82.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.273","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":164.54,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.283","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":190.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:11.292","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":149.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:11.302","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":146.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.1503725,"monotonic":2349768.616877695,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.229","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.241","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.253","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.263","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.273","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":115.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.283","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.292","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":123.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.301","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":99.28,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.1506152,"monotonic":2349769.61712042,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.230","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.242","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.96,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.253","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.263","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.274","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.283","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.293","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.302","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.1514544,"monotonic":2349770.617959667,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.276","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.288","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.299","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.310","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.320","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.330","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.340","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.349","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.84,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.1522532,"monotonic":2349771.618758411,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.231","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.243","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.255","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.265","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.275","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.285","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.294","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:15.303","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.1530979,"monotonic":2349772.619602564,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.256","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.267","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.277","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.288","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.297","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.306","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.316","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.324","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.45,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.153518,"monotonic":2349773.620023016,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.256","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.268","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.67,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.279","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.290","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.300","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.310","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.319","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.329","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.154332,"monotonic":2349774.620837136,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.276","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.288","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.299","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.309","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.319","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.329","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.338","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.347","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.155133,"monotonic":2349775.62163787,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.234","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.246","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.91,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.257","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.268","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.278","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:19.288","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:19.297","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:19.307","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.1559722,"monotonic":2349776.62247757,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527194649633,"port_rcv_data":294749689564836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883193253173,"port_rcv_data":1193269032352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.258","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.269","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.51,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.280","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.290","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.300","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.310","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.319","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.329","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.1567907,"monotonic":2349777.623296142,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.256","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.268","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.279","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.290","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.300","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.96,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.310","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:21.320","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:21.330","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.74,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.15761,"monotonic":2349778.624115305,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.237","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.249","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.03,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.260","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.271","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.281","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.01,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:22.290","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.300","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":82.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.309","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.1584535,"monotonic":2349779.624958666,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.255","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.267","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.278","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.289","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.299","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.309","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.318","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.327","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.57,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.1592796,"monotonic":2349780.625784711,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.259","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.271","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.71,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.282","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.293","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.303","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.313","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:24.323","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:24.332","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.1601,"monotonic":2349781.626605285,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.239","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.251","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.262","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.273","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.283","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.292","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.302","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.311","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.1609426,"monotonic":2349782.627447872,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.240","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.252","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.263","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.274","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.284","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.294","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.304","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.313","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.161783,"monotonic":2349783.628288424,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.276","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.288","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.23,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.299","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.310","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.320","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.329","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.339","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:27.348","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.162591,"monotonic":2349784.629096244,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.259","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.271","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.282","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.293","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.303","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.313","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.323","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.332","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.1634064,"monotonic":2349785.629912175,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.241","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.254","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.265","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.275","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.285","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.42,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.294","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:29.304","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:29.313","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.1642528,"monotonic":2349786.630757619,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.267","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.279","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.290","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.301","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.311","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.321","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.331","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.340","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.57,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.1650693,"monotonic":2349787.63157444,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.267","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.279","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.42,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.290","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.301","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.311","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.321","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.330","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.339","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.1658938,"monotonic":2349788.632398724,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.267","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.278","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.18,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.289","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.83,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.300","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.310","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.320","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.329","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.339","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.1667159,"monotonic":2349789.633220999,"hcas":{"mlx5_0":{"port_xmit_data":294433611345479,"port_rcv_data":294781475995064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817278398386,"port_rcv_data":1232427583935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527279251105,"port_rcv_data":294749689811694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883277814502,"port_rcv_data":1193269265098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.246","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.258","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.9,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.269","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.280","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.290","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.300","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.309","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.319","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.1675727,"monotonic":2349790.634077636,"hcas":{"mlx5_0":{"port_xmit_data":294433726575326,"port_rcv_data":294781476117382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393627378,"port_rcv_data":1232427706775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.315","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.327","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":81.18,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.338","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.348","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.359","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.368","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:34.378","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:34.387","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.168346,"monotonic":2349791.634850654,"hcas":{"mlx5_0":{"port_xmit_data":294433726575326,"port_rcv_data":294781476117382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393627378,"port_rcv_data":1232427706775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.272","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.283","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":81.29,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.294","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.305","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.315","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.324","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.334","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.342","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.1691618,"monotonic":2349792.635666906,"hcas":{"mlx5_0":{"port_xmit_data":294433726575326,"port_rcv_data":294781476117382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393627378,"port_rcv_data":1232427706775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.248","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.259","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.03,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.270","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.281","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.290","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.299","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:36.308","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:36.317","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.1700056,"monotonic":2349793.636510648,"hcas":{"mlx5_0":{"port_xmit_data":294433726575326,"port_rcv_data":294781476117382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393627378,"port_rcv_data":1232427706775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.248","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.260","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.51,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.271","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":77.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.281","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.290","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.299","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.308","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.317","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.170852,"monotonic":2349794.637356797,"hcas":{"mlx5_0":{"port_xmit_data":294433726575326,"port_rcv_data":294781476117382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393627378,"port_rcv_data":1232427706775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.316","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.327","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":81.09,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.338","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.348","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.357","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.43,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.367","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.375","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.384","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.1716318,"monotonic":2349795.638137125,"hcas":{"mlx5_0":{"port_xmit_data":294433726575326,"port_rcv_data":294781476117382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393627378,"port_rcv_data":1232427706775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.315","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.326","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":81.04,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.337","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.347","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.357","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.366","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.376","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":81.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.385","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.1724145,"monotonic":2349796.638919306,"hcas":{"mlx5_0":{"port_xmit_data":294433726575326,"port_rcv_data":294781476117382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393627378,"port_rcv_data":1232427706775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.276","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.287","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.86,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.299","gpu_util_pct":0.0,"memory_used_mib":65433.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.309","gpu_util_pct":0.0,"memory_used_mib":65427.0,"power_w":80.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.319","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.329","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:40.339","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.348","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.1732273,"monotonic":2349797.639732235,"hcas":{"mlx5_0":{"port_xmit_data":294433726740646,"port_rcv_data":294781535780807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817393829629,"port_rcv_data":1232487407023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.280","gpu_util_pct":96.0,"memory_used_mib":65651.0,"power_w":109.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.292","gpu_util_pct":97.0,"memory_used_mib":65651.0,"power_w":115.69,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.304","gpu_util_pct":99.0,"memory_used_mib":65545.0,"power_w":111.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.316","gpu_util_pct":100.0,"memory_used_mib":65651.0,"power_w":112.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.328","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:41.337","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.346","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.355","gpu_util_pct":0.0,"memory_used_mib":70023.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.1740494,"monotonic":2349798.640553999,"hcas":{"mlx5_0":{"port_xmit_data":294433841764228,"port_rcv_data":294781592280487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817508863956,"port_rcv_data":1232543887323,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527395308855,"port_rcv_data":294749689935144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883393871566,"port_rcv_data":1193269388211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.281","gpu_util_pct":100.0,"memory_used_mib":65651.0,"power_w":277.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.293","gpu_util_pct":100.0,"memory_used_mib":65651.0,"power_w":303.01,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.306","gpu_util_pct":100.0,"memory_used_mib":65545.0,"power_w":306.44,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.317","gpu_util_pct":100.0,"memory_used_mib":65651.0,"power_w":305.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:42.328","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":153.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.339","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":143.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.350","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":154.94,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.360","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":162.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.1748526,"monotonic":2349799.641357654,"hcas":{"mlx5_0":{"port_xmit_data":294433841892038,"port_rcv_data":294781651904944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817508997318,"port_rcv_data":1232603518248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527599687105,"port_rcv_data":294749690300598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883598247296,"port_rcv_data":1193269751145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.256","gpu_util_pct":88.0,"memory_used_mib":65651.0,"power_w":300.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.268","gpu_util_pct":82.0,"memory_used_mib":65651.0,"power_w":310.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.280","gpu_util_pct":54.0,"memory_used_mib":65769.0,"power_w":303.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.290","gpu_util_pct":0.0,"memory_used_mib":65651.0,"power_w":281.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.301","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":291.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.312","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":303.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.322","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":291.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.333","gpu_util_pct":100.0,"memory_used_mib":70135.0,"power_w":303.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.1756997,"monotonic":2349800.642204554,"hcas":{"mlx5_0":{"port_xmit_data":294434036815857,"port_rcv_data":294781735891089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817705496953,"port_rcv_data":1232692682400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294527772654003,"port_rcv_data":294749690600333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1883776728797,"port_rcv_data":1193270052724,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.322","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":252.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.334","gpu_util_pct":98.0,"memory_used_mib":65875.0,"power_w":294.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.347","gpu_util_pct":100.0,"memory_used_mib":65769.0,"power_w":275.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.358","gpu_util_pct":93.0,"memory_used_mib":65875.0,"power_w":344.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.369","gpu_util_pct":98.0,"memory_used_mib":70471.0,"power_w":275.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.379","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":283.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.388","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":245.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.398","gpu_util_pct":91.0,"memory_used_mib":70471.0,"power_w":251.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.1764705,"monotonic":2349801.642975552,"hcas":{"mlx5_0":{"port_xmit_data":294434072083792,"port_rcv_data":294781800800487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817739180168,"port_rcv_data":1232752389668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294528092922242,"port_rcv_data":294749691165985,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1884091459597,"port_rcv_data":1193270621074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.284","gpu_util_pct":36.0,"memory_used_mib":65875.0,"power_w":206.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.297","gpu_util_pct":41.0,"memory_used_mib":65875.0,"power_w":227.37,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.309","gpu_util_pct":67.0,"memory_used_mib":65769.0,"power_w":230.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.320","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":257.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:45.331","gpu_util_pct":94.0,"memory_used_mib":70471.0,"power_w":311.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:45.341","gpu_util_pct":96.0,"memory_used_mib":70471.0,"power_w":323.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:45.351","gpu_util_pct":93.0,"memory_used_mib":70471.0,"power_w":318.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.360","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":352.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.1772833,"monotonic":2349802.643788044,"hcas":{"mlx5_0":{"port_xmit_data":294434187146052,"port_rcv_data":294781860591935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817854231027,"port_rcv_data":1232812171495,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294528136907109,"port_rcv_data":294749691295023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1884135420077,"port_rcv_data":1193270744686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.280","gpu_util_pct":99.0,"memory_used_mib":65875.0,"power_w":249.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.294","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":281.22,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.305","gpu_util_pct":100.0,"memory_used_mib":65769.0,"power_w":269.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.316","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":262.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.327","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":255.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.338","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":251.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.348","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":258.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.357","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":240.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.1780992,"monotonic":2349803.644603918,"hcas":{"mlx5_0":{"port_xmit_data":294434302041497,"port_rcv_data":294781866232624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817969126486,"port_rcv_data":1232817799526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294528341262808,"port_rcv_data":294749691638127,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1884339777145,"port_rcv_data":1193271089004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.278","gpu_util_pct":97.0,"memory_used_mib":65875.0,"power_w":258.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.290","gpu_util_pct":98.0,"memory_used_mib":65875.0,"power_w":279.45,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.302","gpu_util_pct":98.0,"memory_used_mib":65769.0,"power_w":245.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.313","gpu_util_pct":31.0,"memory_used_mib":65875.0,"power_w":240.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.323","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":212.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.334","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":226.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.345","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":226.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.356","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":249.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.1789098,"monotonic":2349804.645414532,"hcas":{"mlx5_0":{"port_xmit_data":294434302186607,"port_rcv_data":294781925874516,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1817969276910,"port_rcv_data":1232877446723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294528545576168,"port_rcv_data":294749691941792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1884544116887,"port_rcv_data":1193271420658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.280","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":161.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.292","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":171.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.305","gpu_util_pct":100.0,"memory_used_mib":65769.0,"power_w":174.95,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.317","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":182.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:48.328","gpu_util_pct":56.0,"memory_used_mib":70471.0,"power_w":266.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.337","gpu_util_pct":87.0,"memory_used_mib":70471.0,"power_w":298.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.347","gpu_util_pct":61.0,"memory_used_mib":70471.0,"power_w":281.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.356","gpu_util_pct":0.0,"memory_used_mib":70471.0,"power_w":284.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.179735,"monotonic":2349805.646239724,"hcas":{"mlx5_0":{"port_xmit_data":294434532410684,"port_rcv_data":294781983002867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818199509595,"port_rcv_data":1232934552444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294528553877281,"port_rcv_data":294749691973414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1884552400100,"port_rcv_data":1193271453002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.287","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":162.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.299","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":165.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.311","gpu_util_pct":98.0,"memory_used_mib":65769.0,"power_w":154.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.323","gpu_util_pct":99.0,"memory_used_mib":65875.0,"power_w":153.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.333","gpu_util_pct":98.0,"memory_used_mib":70471.0,"power_w":106.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.343","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":111.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.355","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":116.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.364","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":145.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.1800854,"monotonic":2349806.646590549,"hcas":{"mlx5_0":{"port_xmit_data":294434532558943,"port_rcv_data":294782042645491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818199671414,"port_rcv_data":1232994209083,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294528874274084,"port_rcv_data":294749692430211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1884872804116,"port_rcv_data":1193271914745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.287","gpu_util_pct":97.0,"memory_used_mib":65875.0,"power_w":296.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.299","gpu_util_pct":96.0,"memory_used_mib":65875.0,"power_w":301.08,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.312","gpu_util_pct":97.0,"memory_used_mib":65769.0,"power_w":302.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.323","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":313.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.334","gpu_util_pct":98.0,"memory_used_mib":70471.0,"power_w":256.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.346","gpu_util_pct":97.0,"memory_used_mib":70471.0,"power_w":282.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.356","gpu_util_pct":97.0,"memory_used_mib":70471.0,"power_w":257.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.366","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":249.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.1808968,"monotonic":2349807.647401943,"hcas":{"mlx5_0":{"port_xmit_data":294434647618494,"port_rcv_data":294782102433850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818314729596,"port_rcv_data":1233053995698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294528959605742,"port_rcv_data":294749692607645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1884958102747,"port_rcv_data":1193272089784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.301","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":246.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.313","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":257.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.325","gpu_util_pct":100.0,"memory_used_mib":65769.0,"power_w":239.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:51.336","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":253.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:51.347","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":324.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.357","gpu_util_pct":98.0,"memory_used_mib":70471.0,"power_w":298.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.368","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":318.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.380","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":301.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.1816976,"monotonic":2349808.648202062,"hcas":{"mlx5_0":{"port_xmit_data":294434825753059,"port_rcv_data":294782132639532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818492873324,"port_rcv_data":1233084164382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294529163938525,"port_rcv_data":294749692924962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1885162449325,"port_rcv_data":1193272421226,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.283","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":239.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.295","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":260.3,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.308","gpu_util_pct":100.0,"memory_used_mib":65769.0,"power_w":225.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.320","gpu_util_pct":99.0,"memory_used_mib":65875.0,"power_w":221.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.331","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":185.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.341","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":192.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.350","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":216.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.360","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":207.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.182508,"monotonic":2349809.649013106,"hcas":{"mlx5_0":{"port_xmit_data":294434825938783,"port_rcv_data":294782192319791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818493056391,"port_rcv_data":1233143842145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294529355744617,"port_rcv_data":294749693152306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1885355327789,"port_rcv_data":1193272652155,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.285","gpu_util_pct":98.0,"memory_used_mib":65875.0,"power_w":214.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.297","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":230.24,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.310","gpu_util_pct":100.0,"memory_used_mib":65993.0,"power_w":207.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.322","gpu_util_pct":98.0,"memory_used_mib":65875.0,"power_w":217.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.333","gpu_util_pct":74.0,"memory_used_mib":70471.0,"power_w":275.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.344","gpu_util_pct":44.0,"memory_used_mib":70471.0,"power_w":272.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.355","gpu_util_pct":68.0,"memory_used_mib":70471.0,"power_w":276.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.365","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":288.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.183322,"monotonic":2349810.649826801,"hcas":{"mlx5_0":{"port_xmit_data":294434940982217,"port_rcv_data":294782251363448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818608091091,"port_rcv_data":1233202870548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294529477300224,"port_rcv_data":294749693457214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1885475769395,"port_rcv_data":1193272958118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.301","gpu_util_pct":28.0,"memory_used_mib":65875.0,"power_w":237.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.313","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":241.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.324","gpu_util_pct":60.0,"memory_used_mib":65993.0,"power_w":234.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.336","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":253.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.347","gpu_util_pct":52.0,"memory_used_mib":70471.0,"power_w":238.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.358","gpu_util_pct":22.0,"memory_used_mib":70471.0,"power_w":234.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.368","gpu_util_pct":46.0,"memory_used_mib":70471.0,"power_w":229.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.377","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":234.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.1841295,"monotonic":2349811.650634149,"hcas":{"mlx5_0":{"port_xmit_data":294434958563446,"port_rcv_data":294782311069410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818625661734,"port_rcv_data":1233262566441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294529681656150,"port_rcv_data":294749693800121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1885680144673,"port_rcv_data":1193273319189,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.290","gpu_util_pct":97.0,"memory_used_mib":65875.0,"power_w":286.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.304","gpu_util_pct":96.0,"memory_used_mib":65875.0,"power_w":293.89,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.315","gpu_util_pct":97.0,"memory_used_mib":65993.0,"power_w":288.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.326","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":299.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:55.337","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":255.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.347","gpu_util_pct":98.0,"memory_used_mib":70471.0,"power_w":261.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.358","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":252.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.368","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":241.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.1849408,"monotonic":2349812.651445581,"hcas":{"mlx5_0":{"port_xmit_data":294435073626489,"port_rcv_data":294782369357752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818740716870,"port_rcv_data":1233320801598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294529788270625,"port_rcv_data":294749694070032,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1885786728308,"port_rcv_data":1193273567622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.291","gpu_util_pct":99.0,"memory_used_mib":65875.0,"power_w":256.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.303","gpu_util_pct":99.0,"memory_used_mib":65875.0,"power_w":251.79,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.314","gpu_util_pct":99.0,"memory_used_mib":65993.0,"power_w":213.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.325","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":244.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.336","gpu_util_pct":97.0,"memory_used_mib":70471.0,"power_w":285.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.348","gpu_util_pct":97.0,"memory_used_mib":70471.0,"power_w":289.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.357","gpu_util_pct":98.0,"memory_used_mib":70471.0,"power_w":279.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.367","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":292.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.185535,"monotonic":2349813.652039324,"hcas":{"mlx5_0":{"port_xmit_data":294435190807030,"port_rcv_data":294782429137629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818857873071,"port_rcv_data":1233380557512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294529992639167,"port_rcv_data":294749694425222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1885991100255,"port_rcv_data":1193273927040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.271","gpu_util_pct":97.0,"memory_used_mib":65875.0,"power_w":256.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.284","gpu_util_pct":97.0,"memory_used_mib":65875.0,"power_w":270.33,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.296","gpu_util_pct":97.0,"memory_used_mib":65993.0,"power_w":271.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.307","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":276.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.318","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":258.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.329","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":242.38,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.339","gpu_util_pct":99.0,"memory_used_mib":70471.0,"power_w":238.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.348","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":231.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.1863718,"monotonic":2349814.652876699,"hcas":{"mlx5_0":{"port_xmit_data":294435305863330,"port_rcv_data":294782488922059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1818972932998,"port_rcv_data":1233440346103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530197708732,"port_rcv_data":294749694792981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886196116037,"port_rcv_data":1193274284991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.324","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":269.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.336","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":284.06,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.347","gpu_util_pct":100.0,"memory_used_mib":65993.0,"power_w":254.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.359","gpu_util_pct":100.0,"memory_used_mib":65875.0,"power_w":235.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:58.370","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":314.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:58.381","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":301.73,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:58.392","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":304.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:58.402","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":313.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.1871517,"monotonic":2349815.653656372,"hcas":{"mlx5_0":{"port_xmit_data":294435420746807,"port_rcv_data":294782489728613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819087815993,"port_rcv_data":1233441109954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530402059343,"port_rcv_data":294749695132397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886400460361,"port_rcv_data":1193274618488,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.290","gpu_util_pct":13.0,"memory_used_mib":65879.0,"power_w":245.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.302","gpu_util_pct":8.0,"memory_used_mib":65879.0,"power_w":240.18,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.313","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":208.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.324","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":207.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.334","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":247.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.345","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":242.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.355","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":240.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.365","gpu_util_pct":100.0,"memory_used_mib":70471.0,"power_w":240.93,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.1879787,"monotonic":2349816.654483306,"hcas":{"mlx5_0":{"port_xmit_data":294435485712861,"port_rcv_data":294782489801661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819152781425,"port_rcv_data":1233441182122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530606379317,"port_rcv_data":294749695442883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886604793608,"port_rcv_data":1193274942955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.289","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.301","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.313","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.324","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.334","gpu_util_pct":53.0,"memory_used_mib":70475.0,"power_w":225.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.344","gpu_util_pct":82.0,"memory_used_mib":70475.0,"power_w":196.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.354","gpu_util_pct":57.0,"memory_used_mib":70475.0,"power_w":232.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.364","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":223.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.1888058,"monotonic":2349817.655310614,"hcas":{"mlx5_0":{"port_xmit_data":294435606327404,"port_rcv_data":294782489930060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819273395345,"port_rcv_data":1233441309966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530673204633,"port_rcv_data":294749695524768,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886671580440,"port_rcv_data":1193275024199,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.289","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.301","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.62,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.312","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.322","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.332","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.341","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:01.350","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":85.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.359","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.189637,"monotonic":2349818.656141789,"hcas":{"mlx5_0":{"port_xmit_data":294435606332674,"port_rcv_data":294782489931838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819273400025,"port_rcv_data":1233441311206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530794642483,"port_rcv_data":294749695653977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886793017752,"port_rcv_data":1193275152772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.305","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.317","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.76,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.328","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.339","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.350","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.360","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.369","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.379","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.1903052,"monotonic":2349819.65680994,"hcas":{"mlx5_0":{"port_xmit_data":294435725147624,"port_rcv_data":294782490056858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819392214353,"port_rcv_data":1233441434726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530794647733,"port_rcv_data":294749695655839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886793022432,"port_rcv_data":1193275154012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.288","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.300","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.64,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.311","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.321","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.331","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.340","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.349","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.358","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.1911328,"monotonic":2349820.657637666,"hcas":{"mlx5_0":{"port_xmit_data":294435725153029,"port_rcv_data":294782490058719,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819392219033,"port_rcv_data":1233441435966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530914290750,"port_rcv_data":294749695783244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886912664832,"port_rcv_data":1193275280631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.306","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.317","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.4,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.328","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.338","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.348","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.358","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.368","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.378","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.191943,"monotonic":2349821.658447749,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.317","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.328","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.49,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.339","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.350","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.360","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:05.370","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.379","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.388","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.192741,"monotonic":2349822.659246016,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.340","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.351","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.362","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.373","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.383","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.392","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.401","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.1935227,"monotonic":2349823.660027808,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.292","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.31,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.303","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.314","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.324","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.334","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.344","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.353","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.362","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.194348,"monotonic":2349824.660853268,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.332","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.343","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.72,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.354","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.364","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:08.374","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.383","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.392","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.401","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.195134,"monotonic":2349825.661639108,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.274","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.86,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.285","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.296","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.306","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.316","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.325","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.334","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.343","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.195982,"monotonic":2349826.662487005,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.295","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.306","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.74,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.317","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.327","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.337","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.346","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.356","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.365","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.1968055,"monotonic":2349827.663310514,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.323","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.337","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.348","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.359","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.369","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.378","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:11.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.1975293,"monotonic":2349828.664033713,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.300","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.312","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.323","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.333","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:12.343","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:12.353","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.362","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.371","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.198347,"monotonic":2349829.664852272,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.277","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.289","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.26,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.299","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.310","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.320","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.329","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.338","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.346","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.199191,"monotonic":2349830.665696323,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.295","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.307","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.81,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.317","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.328","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.338","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.347","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.356","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.365","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.2000175,"monotonic":2349831.66652264,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.300","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.311","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.63,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.322","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.333","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.343","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:15.352","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:15.361","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.370","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.200838,"monotonic":2349832.667343261,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.317","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.328","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.339","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.350","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.360","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.369","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.378","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.2016456,"monotonic":2349833.668150966,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.336","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.347","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.358","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.378","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.405","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.2024508,"monotonic":2349834.668955667,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.344","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.355","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.99,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.366","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.2032616,"monotonic":2349835.669766411,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.304","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.316","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.96,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.326","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.337","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.346","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.356","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.365","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.373","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.2041073,"monotonic":2349836.670612082,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.301","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.312","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.18,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.323","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.333","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.343","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:20.352","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.361","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.369","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.2049558,"monotonic":2349837.671460742,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.348","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.360","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.28,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.370","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.381","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.391","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.400","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.409","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.418","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.2050662,"monotonic":2349838.671571038,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.354","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.365","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.15,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.376","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.387","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.397","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:22.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.416","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.425","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.20584,"monotonic":2349839.672345346,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.353","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.365","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.376","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.387","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.397","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.77,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.416","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.425","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.2066128,"monotonic":2349840.673118115,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.344","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.367","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:24.388","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.397","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.415","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.2073963,"monotonic":2349841.673901833,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.363","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.375","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.385","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.395","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.404","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.422","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.2081733,"monotonic":2349842.674678762,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.379","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.390","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:26.400","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.409","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.418","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.427","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.2088923,"monotonic":2349843.675397009,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.348","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.359","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.370","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.381","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.391","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.401","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.418","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.209538,"monotonic":2349844.67604299,"hcas":{"mlx5_0":{"port_xmit_data":294435733582500,"port_rcv_data":294782490077657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400647881,"port_rcv_data":1233441454550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.288","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.300","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:28.311","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:28.321","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:28.330","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:28.340","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.348","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.357","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.2100916,"monotonic":2349845.676596487,"hcas":{"mlx5_0":{"port_xmit_data":294435733762540,"port_rcv_data":294782549756097,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819400847538,"port_rcv_data":1233501152653,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.330","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.343","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":110.64,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.354","gpu_util_pct":1.0,"memory_used_mib":65997.0,"power_w":106.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.366","gpu_util_pct":94.0,"memory_used_mib":65879.0,"power_w":109.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.377","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.386","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.404","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.57,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.2108872,"monotonic":2349846.677392385,"hcas":{"mlx5_0":{"port_xmit_data":294435848796097,"port_rcv_data":294782608755494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819515886739,"port_rcv_data":1233560144185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294530923549451,"port_rcv_data":294749695804972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1886921921752,"port_rcv_data":1193275300378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.312","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":278.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.324","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":289.7,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.335","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":271.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.346","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":279.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:30.357","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":129.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:30.367","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":139.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.379","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":137.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.388","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":125.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.2117124,"monotonic":2349847.678217222,"hcas":{"mlx5_0":{"port_xmit_data":294435848796097,"port_rcv_data":294782608755494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819515886739,"port_rcv_data":1233560144185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531127934383,"port_rcv_data":294749696177573,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887126318314,"port_rcv_data":1193275684548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.312","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":226.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.324","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":215.48,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.335","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":171.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.346","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":210.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.356","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":296.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.367","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":286.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.378","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":294.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.387","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":296.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.2125416,"monotonic":2349848.679046685,"hcas":{"mlx5_0":{"port_xmit_data":294435848796097,"port_rcv_data":294782608755494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819515886739,"port_rcv_data":1233560144185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531216255928,"port_rcv_data":294749696404471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887214628577,"port_rcv_data":1193275919529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.316","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":107.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.328","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":111.48,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.339","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":83.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:32.349","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:32.359","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":187.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.369","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":154.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.378","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":162.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":173.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.2133758,"monotonic":2349849.679880836,"hcas":{"mlx5_0":{"port_xmit_data":294435848796097,"port_rcv_data":294782608755494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819515886739,"port_rcv_data":1233560144185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531216255928,"port_rcv_data":294749696404471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887214628577,"port_rcv_data":1193275919529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.340","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.363","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.374","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.384","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":105.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.394","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.403","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.412","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":103.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.2135372,"monotonic":2349850.680042687,"hcas":{"mlx5_0":{"port_xmit_data":294435848796097,"port_rcv_data":294782608755494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819515886739,"port_rcv_data":1233560144185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531216255928,"port_rcv_data":294749696404471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887214628577,"port_rcv_data":1193275919529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.311","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.323","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.334","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.345","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:34.354","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.364","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.373","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.382","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.2143633,"monotonic":2349851.680868457,"hcas":{"mlx5_0":{"port_xmit_data":294435848796097,"port_rcv_data":294782608755494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819515886739,"port_rcv_data":1233560144185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531216255928,"port_rcv_data":294749696404471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887214628577,"port_rcv_data":1193275919529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.315","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.326","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.338","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.348","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.358","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.368","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.377","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.386","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.2151916,"monotonic":2349852.681696329,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.317","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.329","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.36,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.340","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.350","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.360","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.370","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.379","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.388","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.2160118,"monotonic":2349853.68251656,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.315","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.327","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.62,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.338","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.348","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.357","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.366","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.375","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.384","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.2168365,"monotonic":2349854.683341394,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.367","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.378","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:38.389","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:38.398","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.407","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.416","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.425","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.2175484,"monotonic":2349855.684052827,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.363","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.374","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.384","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.394","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.403","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.412","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.2183425,"monotonic":2349856.684847754,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.357","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.380","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.390","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.401","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.219126,"monotonic":2349857.685631146,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:41.379","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:41.390","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.400","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.2199125,"monotonic":2349858.686417808,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.363","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.25,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.375","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.385","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.395","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.405","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.423","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.2207122,"monotonic":2349859.687217261,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.344","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.367","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:43.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:43.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.415","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.2215266,"monotonic":2349860.688031872,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.367","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.379","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.389","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.400","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.409","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.418","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.2223167,"monotonic":2349861.688821961,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.360","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.372","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.383","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.394","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.405","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.424","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.433","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.2231026,"monotonic":2349862.689607642,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.364","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.376","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.387","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.398","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.408","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.418","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.427","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.436","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.2238941,"monotonic":2349863.690399117,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.391","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.401","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.411","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.420","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.429","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.438","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.2246752,"monotonic":2349864.69118011,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.345","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.357","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.367","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:48.378","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:48.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.397","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.2254808,"monotonic":2349865.691985746,"hcas":{"mlx5_0":{"port_xmit_data":294435969410742,"port_rcv_data":294782608886795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636500659,"port_rcv_data":1233560273347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.324","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.335","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:49.346","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.357","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.367","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.377","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.386","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.395","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.2263067,"monotonic":2349866.692811506,"hcas":{"mlx5_0":{"port_xmit_data":294435969604980,"port_rcv_data":294782668579340,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819636696693,"port_rcv_data":1233619967284,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.326","gpu_util_pct":91.0,"memory_used_mib":65879.0,"power_w":95.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.338","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":135.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.351","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":149.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.362","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":105.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.373","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.382","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.391","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.400","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.227127,"monotonic":2349867.693631998,"hcas":{"mlx5_0":{"port_xmit_data":294436084647119,"port_rcv_data":294782727611995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819751746458,"port_rcv_data":1233678988064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531337698703,"port_rcv_data":294749696535791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887336070569,"port_rcv_data":1193276050520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.332","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":283.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.345","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":295.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.358","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":291.84,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.369","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":280.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.381","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":209.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.391","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":208.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.401","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":222.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.412","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":134.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.2279363,"monotonic":2349868.694440858,"hcas":{"mlx5_0":{"port_xmit_data":294436084647119,"port_rcv_data":294782727611995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819751746458,"port_rcv_data":1233678988064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531542084833,"port_rcv_data":294749696909606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887540463101,"port_rcv_data":1193276430690,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.329","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":182.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.341","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":199.08,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.352","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":153.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.362","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":192.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.372","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":302.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.383","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":303.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.393","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":314.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.403","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":294.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.2287443,"monotonic":2349869.695249308,"hcas":{"mlx5_0":{"port_xmit_data":294436084647119,"port_rcv_data":294782727611995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819751746458,"port_rcv_data":1233678988064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531630431408,"port_rcv_data":294749697127915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887628782680,"port_rcv_data":1193276642621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.358","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":108.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.370","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":98.27,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.382","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":97.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.392","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":96.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.402","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":127.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.411","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":118.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":114.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.430","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":134.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.2295403,"monotonic":2349870.696044697,"hcas":{"mlx5_0":{"port_xmit_data":294436084647119,"port_rcv_data":294782727611995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819751746458,"port_rcv_data":1233678988064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531630431408,"port_rcv_data":294749697127915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887628782680,"port_rcv_data":1193276642621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.333","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.344","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:54.356","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.366","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.376","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.386","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":92.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.395","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":86.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.405","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":110.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.2303529,"monotonic":2349871.696857687,"hcas":{"mlx5_0":{"port_xmit_data":294436084647119,"port_rcv_data":294782727611995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819751746458,"port_rcv_data":1233678988064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531630431408,"port_rcv_data":294749697127915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887628782680,"port_rcv_data":1193276642621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.349","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.361","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.372","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.383","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.393","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.403","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.413","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.422","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.231148,"monotonic":2349872.697653092,"hcas":{"mlx5_0":{"port_xmit_data":294436084647319,"port_rcv_data":294782727612143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819751746458,"port_rcv_data":1233678988064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531630431408,"port_rcv_data":294749697127915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887628782680,"port_rcv_data":1193276642621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.371","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.79,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.389","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.397","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.405","gpu_util_pct":1.0,"memory_used_mib":70475.0,"power_w":80.78,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.413","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.429","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.2319462,"monotonic":2349873.698451416,"hcas":{"mlx5_0":{"port_xmit_data":294436205261797,"port_rcv_data":294782727742074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819872360378,"port_rcv_data":1233679117350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.312","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.324","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.17,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.335","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.345","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.355","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.33,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.364","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.373","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.382","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.232789,"monotonic":2349874.699293966,"hcas":{"mlx5_0":{"port_xmit_data":294436205261797,"port_rcv_data":294782727742074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819872360378,"port_rcv_data":1233679117350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.336","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.347","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.358","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.378","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.387","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.405","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.2336066,"monotonic":2349875.700111922,"hcas":{"mlx5_0":{"port_xmit_data":294436205261797,"port_rcv_data":294782727742074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819872360378,"port_rcv_data":1233679117350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.389","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.400","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.411","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.422","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.432","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.442","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.451","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.2343764,"monotonic":2349876.700881755,"hcas":{"mlx5_0":{"port_xmit_data":294436205261797,"port_rcv_data":294782727742074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819872360378,"port_rcv_data":1233679117350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.376","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.388","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.99,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.400","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.410","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.430","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.440","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.450","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.2351465,"monotonic":2349877.701651725,"hcas":{"mlx5_0":{"port_xmit_data":294436205261797,"port_rcv_data":294782727742074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819872360378,"port_rcv_data":1233679117350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.367","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.05,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.379","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:01.389","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.399","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.94,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.409","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.23594,"monotonic":2349878.702445316,"hcas":{"mlx5_0":{"port_xmit_data":294436205261797,"port_rcv_data":294782727742074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819872360378,"port_rcv_data":1233679117350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.315","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.327","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.06,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.339","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.350","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.360","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.370","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.379","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.389","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.2367823,"monotonic":2349879.703287486,"hcas":{"mlx5_0":{"port_xmit_data":294436205437044,"port_rcv_data":294782787415690,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819872546658,"port_rcv_data":1233738801704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.354","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":149.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.368","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":161.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.380","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":160.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:03.392","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":165.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.402","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.412","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.422","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.431","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.237574,"monotonic":2349880.704079285,"hcas":{"mlx5_0":{"port_xmit_data":294436320468610,"port_rcv_data":294782845707670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819987587045,"port_rcv_data":1233797063269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531751873891,"port_rcv_data":294749697259268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887750224672,"port_rcv_data":1193276771860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.354","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":265.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.366","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":275.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.378","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":280.76,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.390","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":287.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.402","gpu_util_pct":86.0,"memory_used_mib":70475.0,"power_w":193.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.412","gpu_util_pct":17.0,"memory_used_mib":70475.0,"power_w":200.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":193.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.431","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":194.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.2383614,"monotonic":2349881.704866554,"hcas":{"mlx5_0":{"port_xmit_data":294436320468610,"port_rcv_data":294782845707670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819987587045,"port_rcv_data":1233797063269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294531956266229,"port_rcv_data":294749697638918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1887954620357,"port_rcv_data":1193277154873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.319","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":104.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.331","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":143.46,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.343","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":130.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.354","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":156.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.364","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":300.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.374","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":289.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.384","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":294.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.393","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":288.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.2391927,"monotonic":2349882.705698102,"hcas":{"mlx5_0":{"port_xmit_data":294436320468610,"port_rcv_data":294782845707670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819987587045,"port_rcv_data":1233797063269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532043515341,"port_rcv_data":294749697863098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888041823171,"port_rcv_data":1193277368968,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.318","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.330","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":94.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.342","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":107.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":101.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.363","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":119.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.372","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":107.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.382","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":99.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.391","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":102.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.2400281,"monotonic":2349883.706533378,"hcas":{"mlx5_0":{"port_xmit_data":294436320468610,"port_rcv_data":294782845707670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819987587045,"port_rcv_data":1233797063269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532043515341,"port_rcv_data":294749697863098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888041823171,"port_rcv_data":1193277368968,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.319","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.331","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.342","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.353","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.363","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":92.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.373","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.383","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.393","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.2408679,"monotonic":2349884.707373136,"hcas":{"mlx5_0":{"port_xmit_data":294436320468610,"port_rcv_data":294782845707670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1819987587045,"port_rcv_data":1233797063269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532043515341,"port_rcv_data":294749697863098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888041823171,"port_rcv_data":1193277368968,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.340","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.363","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.374","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.385","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.395","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.404","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.2416873,"monotonic":2349885.708192484,"hcas":{"mlx5_0":{"port_xmit_data":294436439288408,"port_rcv_data":294782845836417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106406053,"port_rcv_data":1233797191190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.390","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.401","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.09,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:09.412","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:09.423","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.432","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.442","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.451","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.460","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.2424579,"monotonic":2349886.708963006,"hcas":{"mlx5_0":{"port_xmit_data":294436439288408,"port_rcv_data":294782845836417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106406053,"port_rcv_data":1233797191190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.366","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.21,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.388","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.399","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.409","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.437","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.2432485,"monotonic":2349887.709753743,"hcas":{"mlx5_0":{"port_xmit_data":294436439288408,"port_rcv_data":294782845836417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106406053,"port_rcv_data":1233797191190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.322","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.334","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.345","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.355","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.365","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.375","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.384","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.393","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.2440867,"monotonic":2349888.710591985,"hcas":{"mlx5_0":{"port_xmit_data":294436439288408,"port_rcv_data":294782845836417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106406053,"port_rcv_data":1233797191190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.393","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.405","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.416","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.427","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.437","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.447","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.457","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.466","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.2448642,"monotonic":2349889.711369157,"hcas":{"mlx5_0":{"port_xmit_data":294436439288408,"port_rcv_data":294782845836417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106406053,"port_rcv_data":1233797191190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.390","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.402","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.66,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:13.413","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.8,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.424","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.434","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.54,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.444","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.454","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.463","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.2456408,"monotonic":2349890.712145633,"hcas":{"mlx5_0":{"port_xmit_data":294436439288408,"port_rcv_data":294782845836417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106406053,"port_rcv_data":1233797191190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.390","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.402","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.414","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.92,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.425","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.436","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.446","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.455","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.465","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.246412,"monotonic":2349891.712917077,"hcas":{"mlx5_0":{"port_xmit_data":294436439288408,"port_rcv_data":294782845836417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106406053,"port_rcv_data":1233797191190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.369","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.391","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.402","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.412","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.431","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.440","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.2472095,"monotonic":2349892.713714748,"hcas":{"mlx5_0":{"port_xmit_data":294436439468793,"port_rcv_data":294782905514225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820106581095,"port_rcv_data":1233856863020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.348","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":155.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:16.360","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":163.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:16.371","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":167.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.382","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":168.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.392","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.402","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.412","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.2480235,"monotonic":2349893.714528962,"hcas":{"mlx5_0":{"port_xmit_data":294436554503582,"port_rcv_data":294782962788566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820221607440,"port_rcv_data":1233914098124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532163163139,"port_rcv_data":294749697992648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888161470251,"port_rcv_data":1193277499897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.348","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":279.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.361","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":283.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.372","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":277.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.382","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":286.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.391","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":176.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.402","gpu_util_pct":36.0,"memory_used_mib":70475.0,"power_w":177.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.414","gpu_util_pct":60.0,"memory_used_mib":70475.0,"power_w":191.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.424","gpu_util_pct":53.0,"memory_used_mib":70475.0,"power_w":181.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.2488427,"monotonic":2349894.715347545,"hcas":{"mlx5_0":{"port_xmit_data":294436554503582,"port_rcv_data":294782962788566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820221607440,"port_rcv_data":1233914098124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532453257723,"port_rcv_data":294749698577737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888451518554,"port_rcv_data":1193278068862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":111.14,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.364","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":113.34,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:18.375","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":98.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.387","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":110.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.397","gpu_util_pct":1.0,"memory_used_mib":70475.0,"power_w":288.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.408","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":267.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.417","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":264.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.427","gpu_util_pct":63.0,"memory_used_mib":70475.0,"power_w":275.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.2496614,"monotonic":2349895.716166424,"hcas":{"mlx5_0":{"port_xmit_data":294436554503582,"port_rcv_data":294782962788566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820221607440,"port_rcv_data":1233914098124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532453257723,"port_rcv_data":294749698577737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888451518554,"port_rcv_data":1193278068862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.329","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.11,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.341","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.04,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.352","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.363","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":110.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.373","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.383","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":99.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.392","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":87.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.402","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":106.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.250506,"monotonic":2349896.717011186,"hcas":{"mlx5_0":{"port_xmit_data":294436554503582,"port_rcv_data":294782962788566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820221607440,"port_rcv_data":1233914098124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532453257723,"port_rcv_data":294749698577737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888451518554,"port_rcv_data":1193278068862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:20.364","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.16,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:20.375","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.386","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.415","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.425","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.2513263,"monotonic":2349897.717831466,"hcas":{"mlx5_0":{"port_xmit_data":294436554503582,"port_rcv_data":294782962788566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820221607440,"port_rcv_data":1233914098124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532453257723,"port_rcv_data":294749698577737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888451518554,"port_rcv_data":1193278068862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.348","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.360","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.96,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.371","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.382","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.392","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.401","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.411","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.420","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.2521572,"monotonic":2349898.718662031,"hcas":{"mlx5_0":{"port_xmit_data":294436671528500,"port_rcv_data":294782962914385,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820338631536,"port_rcv_data":1233914223550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532571110642,"port_rcv_data":294749698705568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888569370722,"port_rcv_data":1193278196380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.353","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.364","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.86,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.375","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.7,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.386","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.415","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.424","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.2529836,"monotonic":2349899.719488869,"hcas":{"mlx5_0":{"port_xmit_data":294436671528500,"port_rcv_data":294782962914385,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820338631536,"port_rcv_data":1233914223550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532571110642,"port_rcv_data":294749698705568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888569370722,"port_rcv_data":1193278196380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.369","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.23,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.381","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.393","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.404","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.54,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.413","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.422","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.431","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.2537982,"monotonic":2349900.720303251,"hcas":{"mlx5_0":{"port_xmit_data":294436786745574,"port_rcv_data":294783079760290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820453843662,"port_rcv_data":1234031045611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532571110642,"port_rcv_data":294749698705568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888569370722,"port_rcv_data":1193278196380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.355","gpu_util_pct":64.0,"memory_used_mib":65879.0,"power_w":208.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.368","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":216.44,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.379","gpu_util_pct":97.0,"memory_used_mib":65997.0,"power_w":221.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.391","gpu_util_pct":99.0,"memory_used_mib":65879.0,"power_w":212.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.402","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":126.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.413","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":121.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.424","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":125.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.433","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":124.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.2546165,"monotonic":2349901.721121331,"hcas":{"mlx5_0":{"port_xmit_data":294436786745574,"port_rcv_data":294783079760290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820453843662,"port_rcv_data":1234031045611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532775496610,"port_rcv_data":294749699078631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888773755601,"port_rcv_data":1193278568820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.353","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":240.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.365","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":254.62,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.376","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":219.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.387","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":215.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.397","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":299.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.408","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":304.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.419","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":304.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.429","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":312.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.2554343,"monotonic":2349902.721939313,"hcas":{"mlx5_0":{"port_xmit_data":294436786745574,"port_rcv_data":294783079760290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820453843662,"port_rcv_data":1234031045611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532861046703,"port_rcv_data":294749699289785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888859259664,"port_rcv_data":1193278753476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":99.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":99.29,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.379","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":82.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.390","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.400","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":174.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":184.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":180.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":172.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.2562535,"monotonic":2349903.722758513,"hcas":{"mlx5_0":{"port_xmit_data":294436786745574,"port_rcv_data":294783079760290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820453843662,"port_rcv_data":1234031045611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532861046703,"port_rcv_data":294749699289785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888859259664,"port_rcv_data":1193278753476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.335","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.347","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.358","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.367","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.375","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":102.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.382","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":106.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.390","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":106.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.398","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.2571032,"monotonic":2349904.723608373,"hcas":{"mlx5_0":{"port_xmit_data":294436786745574,"port_rcv_data":294783079760290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820453843662,"port_rcv_data":1234031045611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532861046703,"port_rcv_data":294749699289785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888859259664,"port_rcv_data":1193278753476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.356","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.12,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.379","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:28.389","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.399","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.408","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.417","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.426","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.257518,"monotonic":2349905.724023242,"hcas":{"mlx5_0":{"port_xmit_data":294436786745574,"port_rcv_data":294783079760290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820453843662,"port_rcv_data":1234031045611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532861046703,"port_rcv_data":294749699289785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888859259664,"port_rcv_data":1193278753476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.355","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.366","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.377","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.387","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.396","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.415","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.424","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.2583544,"monotonic":2349906.724859289,"hcas":{"mlx5_0":{"port_xmit_data":294436903770427,"port_rcv_data":294783079885362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820570867758,"port_rcv_data":1234031169797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532978899084,"port_rcv_data":294749699416644,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888977111832,"port_rcv_data":1193278879909,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.361","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.373","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.32,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.384","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.394","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.404","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.423","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.432","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.2591712,"monotonic":2349907.725676617,"hcas":{"mlx5_0":{"port_xmit_data":294436903940141,"port_rcv_data":294783139552979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820571047214,"port_rcv_data":1234090846692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294532978899622,"port_rcv_data":294749699417234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1888977111832,"port_rcv_data":1193278879909,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.380","gpu_util_pct":58.0,"memory_used_mib":65879.0,"power_w":214.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.393","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":240.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.404","gpu_util_pct":25.0,"memory_used_mib":65997.0,"power_w":224.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.415","gpu_util_pct":13.0,"memory_used_mib":65879.0,"power_w":216.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.424","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.435","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.445","gpu_util_pct":25.0,"memory_used_mib":70475.0,"power_w":83.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.455","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":100.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.2599726,"monotonic":2349908.726477631,"hcas":{"mlx5_0":{"port_xmit_data":294437018981567,"port_rcv_data":294783198484474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820686091596,"port_rcv_data":1234149744428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533183269762,"port_rcv_data":294749699774764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889181495980,"port_rcv_data":1193279251664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.361","gpu_util_pct":96.0,"memory_used_mib":65879.0,"power_w":259.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.372","gpu_util_pct":9.0,"memory_used_mib":65879.0,"power_w":279.14,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:32.383","gpu_util_pct":77.0,"memory_used_mib":65997.0,"power_w":274.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:32.394","gpu_util_pct":66.0,"memory_used_mib":65879.0,"power_w":266.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.404","gpu_util_pct":99.0,"memory_used_mib":70475.0,"power_w":290.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.415","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":302.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.425","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":272.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.435","gpu_util_pct":89.0,"memory_used_mib":70475.0,"power_w":301.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.2604346,"monotonic":2349909.726939862,"hcas":{"mlx5_0":{"port_xmit_data":294437018981567,"port_rcv_data":294783198484474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820686091596,"port_rcv_data":1234149744428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533271459957,"port_rcv_data":294749699995704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889269645306,"port_rcv_data":1193279462613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.363","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.68,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.375","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.89,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.386","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":81.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.396","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":101.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.406","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":238.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.416","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":253.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.425","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":253.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.434","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":244.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.2612586,"monotonic":2349910.727763791,"hcas":{"mlx5_0":{"port_xmit_data":294437018981567,"port_rcv_data":294783198484474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820686091596,"port_rcv_data":1234149744428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533271459957,"port_rcv_data":294749699995704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889269645306,"port_rcv_data":1193279462613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.340","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.352","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.362","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.373","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.383","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":98.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.392","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":107.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.401","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":110.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.2621043,"monotonic":2349911.728609608,"hcas":{"mlx5_0":{"port_xmit_data":294437018981567,"port_rcv_data":294783198484474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820686091596,"port_rcv_data":1234149744428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533271459957,"port_rcv_data":294749699995704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889269645306,"port_rcv_data":1193279462613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.363","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.374","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:35.385","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:35.395","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.405","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.414","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.423","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.432","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.2629268,"monotonic":2349912.72943227,"hcas":{"mlx5_0":{"port_xmit_data":294437018981567,"port_rcv_data":294783198484474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820686091596,"port_rcv_data":1234149744428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533271459957,"port_rcv_data":294749699995704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889269645306,"port_rcv_data":1193279462613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.384","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.395","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.406","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.416","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.425","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.435","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.444","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.453","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.2631404,"monotonic":2349913.729645071,"hcas":{"mlx5_0":{"port_xmit_data":294437018981767,"port_rcv_data":294783198484622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820686091596,"port_rcv_data":1234149744428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533271459957,"port_rcv_data":294749699995704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889269645306,"port_rcv_data":1193279462613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.364","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.376","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.38,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:37.387","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.397","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.408","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.417","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.427","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.436","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.2639642,"monotonic":2349914.73046914,"hcas":{"mlx5_0":{"port_xmit_data":294437139758947,"port_rcv_data":294783258276336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820806882943,"port_rcv_data":1234209550609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533271464850,"port_rcv_data":294749699997209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889269649986,"port_rcv_data":1193279463853,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.387","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":188.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.400","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":173.34,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.411","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":152.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.423","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":158.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.433","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.442","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.451","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.460","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.2647643,"monotonic":2349915.731269294,"hcas":{"mlx5_0":{"port_xmit_data":294437254796381,"port_rcv_data":294783314876798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820921918991,"port_rcv_data":1234266127549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533392902162,"port_rcv_data":294749700125781,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889391087298,"port_rcv_data":1193279594192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.368","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":278.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.383","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":281.7,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.394","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":284.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.405","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":284.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.415","gpu_util_pct":88.0,"memory_used_mib":70475.0,"power_w":239.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.425","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":241.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.435","gpu_util_pct":93.0,"memory_used_mib":70475.0,"power_w":249.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.445","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":242.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.2655838,"monotonic":2349916.732088646,"hcas":{"mlx5_0":{"port_xmit_data":294437254796381,"port_rcv_data":294783314876798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820921918991,"port_rcv_data":1234266127549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533597295364,"port_rcv_data":294749700506301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889595435240,"port_rcv_data":1193279929912,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.386","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":133.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.398","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":137.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.409","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":151.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.420","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":143.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.431","gpu_util_pct":86.0,"memory_used_mib":70475.0,"power_w":286.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.441","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":291.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.450","gpu_util_pct":92.0,"memory_used_mib":70475.0,"power_w":277.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.460","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":279.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.2663834,"monotonic":2349917.732888358,"hcas":{"mlx5_0":{"port_xmit_data":294437254796381,"port_rcv_data":294783314876798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820921918991,"port_rcv_data":1234266127549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533681967720,"port_rcv_data":294749700687003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889680082417,"port_rcv_data":1193280111011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.368","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.08,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.391","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":99.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.402","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":112.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.413","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":106.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.423","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":91.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.432","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":86.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.442","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":112.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.2672024,"monotonic":2349918.733707336,"hcas":{"mlx5_0":{"port_xmit_data":294437254796381,"port_rcv_data":294783314876798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820921918991,"port_rcv_data":1234266127549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533681967720,"port_rcv_data":294749700687003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889680082417,"port_rcv_data":1193280111011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.346","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:42.358","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.96,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:42.370","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.390","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.400","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":91.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":86.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.2680466,"monotonic":2349919.73455167,"hcas":{"mlx5_0":{"port_xmit_data":294437254796381,"port_rcv_data":294783314876798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1820921918991,"port_rcv_data":1234266127549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533681967720,"port_rcv_data":294749700687003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889680082417,"port_rcv_data":1193280111011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.372","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.384","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.395","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.406","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.416","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.426","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.436","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.445","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.268866,"monotonic":2349920.735371134,"hcas":{"mlx5_0":{"port_xmit_data":294437370923587,"port_rcv_data":294783315001173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821038045439,"port_rcv_data":1234266252246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533798922453,"port_rcv_data":294749700814011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889797036937,"port_rcv_data":1193280236483,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.369","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.33,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.390","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.401","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.411","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.420","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.429","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.438","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.269694,"monotonic":2349921.736199293,"hcas":{"mlx5_0":{"port_xmit_data":294437370923587,"port_rcv_data":294783315001173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821038045439,"port_rcv_data":1234266252246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533798923024,"port_rcv_data":294749700814634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889797036937,"port_rcv_data":1193280236483,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:45.388","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:45.402","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.71,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:45.413","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.425","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.435","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.445","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.454","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.463","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.2704933,"monotonic":2349922.736998167,"hcas":{"mlx5_0":{"port_xmit_data":294437486137565,"port_rcv_data":294783433878147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821153273922,"port_rcv_data":1234385139170,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294533798923024,"port_rcv_data":294749700814634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1889797036937,"port_rcv_data":1193280236483,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.374","gpu_util_pct":33.0,"memory_used_mib":65879.0,"power_w":215.78,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.389","gpu_util_pct":98.0,"memory_used_mib":65879.0,"power_w":228.58,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.401","gpu_util_pct":65.0,"memory_used_mib":65997.0,"power_w":235.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.415","gpu_util_pct":77.0,"memory_used_mib":65879.0,"power_w":255.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.426","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":141.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.437","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":132.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.449","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":125.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.460","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":138.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.2712917,"monotonic":2349923.737796279,"hcas":{"mlx5_0":{"port_xmit_data":294437486281864,"port_rcv_data":294783493519171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821153442835,"port_rcv_data":1234444805553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534003293976,"port_rcv_data":294749701172589,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890001416324,"port_rcv_data":1193280603229,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.391","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":322.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.404","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":353.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.416","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":344.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.428","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":344.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.439","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":304.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.450","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":284.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.460","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":285.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.470","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":293.7,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.2720811,"monotonic":2349924.738586157,"hcas":{"mlx5_0":{"port_xmit_data":294437601305345,"port_rcv_data":294783552001095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821268476278,"port_rcv_data":1234503280760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534091950145,"port_rcv_data":294749701428978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890090049976,"port_rcv_data":1193280842916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.379","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":272.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.391","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":291.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.403","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":250.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.415","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":256.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.425","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":328.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.435","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":336.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.444","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":335.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.454","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":298.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.272887,"monotonic":2349925.739392254,"hcas":{"mlx5_0":{"port_xmit_data":294437601305345,"port_rcv_data":294783552001095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821268476278,"port_rcv_data":1234503280760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534296316893,"port_rcv_data":294749701785364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890294408497,"port_rcv_data":1193281189370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.375","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":164.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.387","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":150.84,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.398","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":147.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.409","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":159.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.419","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":272.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.429","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":247.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.438","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":272.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.448","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":259.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.2735376,"monotonic":2349926.740042693,"hcas":{"mlx5_0":{"port_xmit_data":294437601305345,"port_rcv_data":294783552001095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821268476278,"port_rcv_data":1234503280760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534383883873,"port_rcv_data":294749702018888,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890381970022,"port_rcv_data":1193281434071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.389","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.72,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.400","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.411","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":134.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.431","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":130.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.441","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":127.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.450","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":121.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.2743514,"monotonic":2349927.74085658,"hcas":{"mlx5_0":{"port_xmit_data":294437601305345,"port_rcv_data":294783552001095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821268476278,"port_rcv_data":1234503280760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534383883873,"port_rcv_data":294749702018888,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890381970022,"port_rcv_data":1193281434071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.376","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.387","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.399","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.409","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":85.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.437","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":85.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.446","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.2751696,"monotonic":2349928.741674801,"hcas":{"mlx5_0":{"port_xmit_data":294437601305545,"port_rcv_data":294783552001243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821268476278,"port_rcv_data":1234503280760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534383883873,"port_rcv_data":294749702018888,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890381970022,"port_rcv_data":1193281434071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:52.397","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.83,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:52.409","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.92,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.420","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.430","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.440","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.449","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.459","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.467","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.2759635,"monotonic":2349929.742468439,"hcas":{"mlx5_0":{"port_xmit_data":294437721915343,"port_rcv_data":294783552129670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821389085518,"port_rcv_data":1234503409999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534383888553,"port_rcv_data":294749702020128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890381974702,"port_rcv_data":1193281435311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.389","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.82,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.400","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.410","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.421","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.430","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.440","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.449","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.2767768,"monotonic":2349930.743281893,"hcas":{"mlx5_0":{"port_xmit_data":294437721920223,"port_rcv_data":294783552131058,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821389090198,"port_rcv_data":1234503411239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534505326481,"port_rcv_data":294749702150818,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890503412014,"port_rcv_data":1193281565774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.417","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.428","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.71,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.439","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.449","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.459","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.468","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.478","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.486","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.277553,"monotonic":2349931.744058507,"hcas":{"mlx5_0":{"port_xmit_data":294437841796304,"port_rcv_data":294783611918632,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821508977478,"port_rcv_data":1234563209987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534505331769,"port_rcv_data":294749702152731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890503416694,"port_rcv_data":1193281567014,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.381","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":165.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.394","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":168.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.406","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":186.82,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.418","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":186.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.430","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.439","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.448","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.457","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.2783544,"monotonic":2349932.744859828,"hcas":{"mlx5_0":{"port_xmit_data":294437956839606,"port_rcv_data":294783671181755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821624020328,"port_rcv_data":1234622439729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534625871625,"port_rcv_data":294749702280389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890623956550,"port_rcv_data":1193281695959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:56.379","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":279.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.391","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":314.6,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.403","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":275.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.414","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":274.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.424","gpu_util_pct":43.0,"memory_used_mib":70475.0,"power_w":242.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.434","gpu_util_pct":76.0,"memory_used_mib":70475.0,"power_w":215.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.444","gpu_util_pct":50.0,"memory_used_mib":70475.0,"power_w":239.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.453","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":250.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.2791765,"monotonic":2349933.745681196,"hcas":{"mlx5_0":{"port_xmit_data":294437956839606,"port_rcv_data":294783671181755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821624020328,"port_rcv_data":1234622439729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534830268487,"port_rcv_data":294749702664615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890828339952,"port_rcv_data":1193282067215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.362","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":158.05,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.374","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":162.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.385","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":147.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.396","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":139.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.406","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":299.7,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.416","gpu_util_pct":96.0,"memory_used_mib":70475.0,"power_w":280.22,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.426","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":275.84,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.435","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":302.27,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.280017,"monotonic":2349934.746521768,"hcas":{"mlx5_0":{"port_xmit_data":294437956839606,"port_rcv_data":294783671181755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821624020328,"port_rcv_data":1234622439729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534918949604,"port_rcv_data":294749702879725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890916993376,"port_rcv_data":1193282279107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:58.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":108.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.392","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":106.41,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.403","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":100.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.414","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.425","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":85.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.435","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.444","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":86.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.454","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":108.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.2808385,"monotonic":2349935.747343444,"hcas":{"mlx5_0":{"port_xmit_data":294437956839606,"port_rcv_data":294783671181755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821624020328,"port_rcv_data":1234622439729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534918949604,"port_rcv_data":294749702879725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890916993376,"port_rcv_data":1193282279107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.380","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.392","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.45,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.403","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.414","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.424","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.434","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.443","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.453","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.2816637,"monotonic":2349936.748168715,"hcas":{"mlx5_0":{"port_xmit_data":294437956839606,"port_rcv_data":294783671181755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821624020328,"port_rcv_data":1234622439729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534918949604,"port_rcv_data":294749702879725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890916993376,"port_rcv_data":1193282279107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.379","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.23,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.391","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.45,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.403","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.413","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.424","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.434","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.443","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.453","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.2824914,"monotonic":2349937.748996186,"hcas":{"mlx5_0":{"port_xmit_data":294437956839606,"port_rcv_data":294783671181755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821624020328,"port_rcv_data":1234622439729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534918949604,"port_rcv_data":294749702879725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890916993376,"port_rcv_data":1193282279107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:01.404","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.416","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.2,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.427","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.438","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.448","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.458","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.468","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.477","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.2832942,"monotonic":2349938.749799218,"hcas":{"mlx5_0":{"port_xmit_data":294437957012230,"port_rcv_data":294783730852153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821624162600,"port_rcv_data":1234682079332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534918949604,"port_rcv_data":294749702879725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890916993376,"port_rcv_data":1193282279107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.364","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":149.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.376","gpu_util_pct":62.0,"memory_used_mib":65879.0,"power_w":163.69,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.387","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":155.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.398","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":158.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.408","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.418","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.427","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.436","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.2841442,"monotonic":2349939.750649009,"hcas":{"mlx5_0":{"port_xmit_data":294438192660449,"port_rcv_data":294783789783667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821859812311,"port_rcv_data":1234741005937,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294534998695201,"port_rcv_data":294749702977136,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1890996968448,"port_rcv_data":1193282383643,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.407","gpu_util_pct":47.0,"memory_used_mib":65879.0,"power_w":271.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.419","gpu_util_pct":94.0,"memory_used_mib":65879.0,"power_w":290.83,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.430","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":263.96,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.441","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":257.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.451","gpu_util_pct":99.0,"memory_used_mib":70475.0,"power_w":274.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.461","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":273.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.470","gpu_util_pct":99.0,"memory_used_mib":70475.0,"power_w":274.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.481","gpu_util_pct":64.0,"memory_used_mib":70475.0,"power_w":249.65,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.2849386,"monotonic":2349940.751443653,"hcas":{"mlx5_0":{"port_xmit_data":294438192663021,"port_rcv_data":294783789784287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821859814419,"port_rcv_data":1234741006510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535332796626,"port_rcv_data":294749703610622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891330848708,"port_rcv_data":1193283025035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.388","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":90.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.400","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":90.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.412","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":82.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.423","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.433","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":271.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.442","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":274.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.452","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":264.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.461","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":260.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.2857592,"monotonic":2349941.752264187,"hcas":{"mlx5_0":{"port_xmit_data":294438192663021,"port_rcv_data":294783789784287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821859814419,"port_rcv_data":1234741006510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535332796626,"port_rcv_data":294749703610622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891330848708,"port_rcv_data":1193283025035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.365","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.377","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.25,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.388","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.399","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.409","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.419","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.429","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.438","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.2866027,"monotonic":2349942.753107837,"hcas":{"mlx5_0":{"port_xmit_data":294438192663021,"port_rcv_data":294783789784287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821859814419,"port_rcv_data":1234741006510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535332796626,"port_rcv_data":294749703610622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891330848708,"port_rcv_data":1193283025035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:06.366","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.78,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.378","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.389","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.400","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.410","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.420","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.430","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.439","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.2874455,"monotonic":2349943.753950313,"hcas":{"mlx5_0":{"port_xmit_data":294438192663021,"port_rcv_data":294783789784287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821859814419,"port_rcv_data":1234741006510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535332796626,"port_rcv_data":294749703610622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891330848708,"port_rcv_data":1193283025035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.367","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.378","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.390","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.400","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.411","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.420","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.430","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.439","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.2882895,"monotonic":2349944.754794555,"hcas":{"mlx5_0":{"port_xmit_data":294438192799039,"port_rcv_data":294783849417682,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821859961996,"port_rcv_data":1234800651898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535332796626,"port_rcv_data":294749703610622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891330848708,"port_rcv_data":1193283025035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.394","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":157.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.407","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":179.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.419","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":164.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.431","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":174.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.442","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.451","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.460","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.469","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.2891035,"monotonic":2349945.755608449,"hcas":{"mlx5_0":{"port_xmit_data":294438307831009,"port_rcv_data":294783907909142,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1821974992202,"port_rcv_data":1234859095883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535332796626,"port_rcv_data":294749703610622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891330848708,"port_rcv_data":1193283025035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.416","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":277.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.429","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":285.67,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.440","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":271.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.451","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":289.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.461","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":188.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.471","gpu_util_pct":19.0,"memory_used_mib":70475.0,"power_w":191.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.482","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":199.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.491","gpu_util_pct":53.0,"memory_used_mib":70475.0,"power_w":184.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.2898972,"monotonic":2349946.756402637,"hcas":{"mlx5_0":{"port_xmit_data":294438427543350,"port_rcv_data":294783908037011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822094703986,"port_rcv_data":1234859223386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535541167605,"port_rcv_data":294749703973493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891549328164,"port_rcv_data":1193283415738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.409","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":108.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.420","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":134.37,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.432","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":128.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.442","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":128.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.451","gpu_util_pct":33.0,"memory_used_mib":70475.0,"power_w":290.39,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.461","gpu_util_pct":57.0,"memory_used_mib":70475.0,"power_w":282.61,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.470","gpu_util_pct":40.0,"memory_used_mib":70475.0,"power_w":260.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.479","gpu_util_pct":90.0,"memory_used_mib":70475.0,"power_w":260.3,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.2907033,"monotonic":2349947.757208557,"hcas":{"mlx5_0":{"port_xmit_data":294438427548030,"port_rcv_data":294783908038251,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822094708666,"port_rcv_data":1234859224626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535745229822,"port_rcv_data":294749704293674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891743282960,"port_rcv_data":1193283752785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.370","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.381","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":86.19,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.392","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":105.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.403","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":103.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.412","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":91.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.422","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.431","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":96.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.440","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.2915485,"monotonic":2349948.758053514,"hcas":{"mlx5_0":{"port_xmit_data":294438427548030,"port_rcv_data":294783908038251,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822094708666,"port_rcv_data":1234859224626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535745229822,"port_rcv_data":294749704293674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891743282960,"port_rcv_data":1193283752785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.392","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.403","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.8,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.414","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.424","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.434","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.444","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.453","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.462","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.29237,"monotonic":2349949.758875233,"hcas":{"mlx5_0":{"port_xmit_data":294438427548030,"port_rcv_data":294783908038251,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822094708666,"port_rcv_data":1234859224626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535745229822,"port_rcv_data":294749704293674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891743282960,"port_rcv_data":1193283752785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:13.395","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.406","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.77,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.417","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.427","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.437","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.446","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.455","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.464","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.2931917,"monotonic":2349950.759696852,"hcas":{"mlx5_0":{"port_xmit_data":294438427548295,"port_rcv_data":294783908038464,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822094708666,"port_rcv_data":1234859224626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535745229822,"port_rcv_data":294749704293674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891743282960,"port_rcv_data":1193283752785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.395","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.406","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.46,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.417","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.427","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.437","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.447","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.456","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.465","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.2940097,"monotonic":2349951.760514942,"hcas":{"mlx5_0":{"port_xmit_data":294438547260344,"port_rcv_data":294783908165731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822214420450,"port_rcv_data":1234859352082,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535745234502,"port_rcv_data":294749704294914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891743287640,"port_rcv_data":1193283754025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.393","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.405","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.79,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.417","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.427","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.438","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.447","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.456","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.465","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.2948303,"monotonic":2349952.761335639,"hcas":{"mlx5_0":{"port_xmit_data":294438662494097,"port_rcv_data":294784027626785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822329657621,"port_rcv_data":1234978815672,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294535865774571,"port_rcv_data":294749704425658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1891863827496,"port_rcv_data":1193283882566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:16.396","gpu_util_pct":65.0,"memory_used_mib":65879.0,"power_w":281.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.408","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":306.23,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.420","gpu_util_pct":97.0,"memory_used_mib":65997.0,"power_w":251.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.431","gpu_util_pct":99.0,"memory_used_mib":65879.0,"power_w":273.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.442","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":156.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.453","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":149.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.463","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":156.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.472","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":178.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.295645,"monotonic":2349953.762150096,"hcas":{"mlx5_0":{"port_xmit_data":294438777424099,"port_rcv_data":294784047878511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822444587188,"port_rcv_data":1234999056013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536070128642,"port_rcv_data":294749704767031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892068199098,"port_rcv_data":1193284241652,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.398","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":345.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.410","gpu_util_pct":93.0,"memory_used_mib":65879.0,"power_w":367.86,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.421","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":300.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.432","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":305.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.442","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":305.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.452","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":291.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.463","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":281.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.473","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":270.8,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.2964573,"monotonic":2349954.762962413,"hcas":{"mlx5_0":{"port_xmit_data":294438777424099,"port_rcv_data":294784047878511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822444587188,"port_rcv_data":1234999056013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536274446636,"port_rcv_data":294749705075221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892272526969,"port_rcv_data":1193284560787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.400","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.412","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":109.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.423","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":83.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.433","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.443","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":240.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.452","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":266.49,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.462","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":245.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.471","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":261.28,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.2972767,"monotonic":2349955.763782005,"hcas":{"mlx5_0":{"port_xmit_data":294438777424099,"port_rcv_data":294784047878511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822444587188,"port_rcv_data":1234999056013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536304652774,"port_rcv_data":294749705152702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892302718474,"port_rcv_data":1193284639019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.395","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.407","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.5,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.418","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.428","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.438","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.447","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.457","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.466","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.2981043,"monotonic":2349956.764609458,"hcas":{"mlx5_0":{"port_xmit_data":294438777424099,"port_rcv_data":294784047878511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822444587188,"port_rcv_data":1234999056013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536304652774,"port_rcv_data":294749705152702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892302718474,"port_rcv_data":1193284639019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.400","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.412","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.423","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.434","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.445","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.455","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.465","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.474","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.2989197,"monotonic":2349957.765424887,"hcas":{"mlx5_0":{"port_xmit_data":294438777424099,"port_rcv_data":294784047878511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822444587188,"port_rcv_data":1234999056013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536304652774,"port_rcv_data":294749705152702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892302718474,"port_rcv_data":1193284639019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:21.399","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.411","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.33,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.422","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.432","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.442","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.452","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.461","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.470","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.2997499,"monotonic":2349958.766249534,"hcas":{"mlx5_0":{"port_xmit_data":294438823548631,"port_rcv_data":294784047937442,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822490710340,"port_rcv_data":1234999112092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536351605157,"port_rcv_data":294749705210716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892349669698,"port_rcv_data":1193284694943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.397","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.408","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.03,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.419","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.430","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.440","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.449","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.459","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.468","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.3005676,"monotonic":2349959.767072816,"hcas":{"mlx5_0":{"port_xmit_data":294438823548631,"port_rcv_data":294784047937442,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822490710340,"port_rcv_data":1234999112092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536351605157,"port_rcv_data":294749705210716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892349669698,"port_rcv_data":1193284694943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.403","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.415","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.426","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.436","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.445","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.455","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.464","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.473","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.3013806,"monotonic":2349960.767886005,"hcas":{"mlx5_0":{"port_xmit_data":294438823732581,"port_rcv_data":294784107619652,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822490886010,"port_rcv_data":1235058786177,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536351605157,"port_rcv_data":294749705210716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892349669698,"port_rcv_data":1193284694943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.425","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":196.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.437","gpu_util_pct":46.0,"memory_used_mib":65879.0,"power_w":200.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.449","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":180.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.461","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":201.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.472","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.483","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.493","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.504","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.3021657,"monotonic":2349961.768671048,"hcas":{"mlx5_0":{"port_xmit_data":294438938764650,"port_rcv_data":294784164027378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822605918793,"port_rcv_data":1235115190398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536555998372,"port_rcv_data":294749705591383,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892554070485,"port_rcv_data":1193285083012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.399","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":271.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.412","gpu_util_pct":42.0,"memory_used_mib":65879.0,"power_w":272.79,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.423","gpu_util_pct":97.0,"memory_used_mib":65997.0,"power_w":276.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.434","gpu_util_pct":97.0,"memory_used_mib":65879.0,"power_w":280.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.444","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":255.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.455","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":247.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.466","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":255.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.476","gpu_util_pct":99.0,"memory_used_mib":70475.0,"power_w":271.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.3029788,"monotonic":2349962.769483956,"hcas":{"mlx5_0":{"port_xmit_data":294438938764650,"port_rcv_data":294784164027378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822605918793,"port_rcv_data":1235115190398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536640408070,"port_rcv_data":294749705791686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892638474632,"port_rcv_data":1193285284090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.405","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":109.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.417","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":105.39,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.428","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":84.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.439","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":86.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.450","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":239.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.460","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":245.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.470","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":238.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.479","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":203.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.3037915,"monotonic":2349963.77029678,"hcas":{"mlx5_0":{"port_xmit_data":294438938764650,"port_rcv_data":294784164027378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822605918793,"port_rcv_data":1235115190398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536640408070,"port_rcv_data":294749705791686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892638474632,"port_rcv_data":1193285284090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.403","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.415","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.427","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.438","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.448","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":107.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.458","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":100.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.467","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":103.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.477","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.3046126,"monotonic":2349964.771117752,"hcas":{"mlx5_0":{"port_xmit_data":294438938764650,"port_rcv_data":294784164027378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822605918793,"port_rcv_data":1235115190398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536640408070,"port_rcv_data":294749705791686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892638474632,"port_rcv_data":1193285284090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.384","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.395","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.407","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.417","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.428","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.438","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.447","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.457","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.305452,"monotonic":2349965.77195718,"hcas":{"mlx5_0":{"port_xmit_data":294438938764650,"port_rcv_data":294784164027378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822605918793,"port_rcv_data":1235115190398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536640408070,"port_rcv_data":294749705791686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892638474632,"port_rcv_data":1193285284090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.403","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.415","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.426","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.436","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.446","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.456","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.466","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.475","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.3062823,"monotonic":2349966.77278732,"hcas":{"mlx5_0":{"port_xmit_data":294439053994399,"port_rcv_data":294784164149474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721147785,"port_rcv_data":1235115312352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.408","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.420","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.431","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.442","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.452","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.462","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.472","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.481","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.3071024,"monotonic":2349967.773607504,"hcas":{"mlx5_0":{"port_xmit_data":294439053994399,"port_rcv_data":294784164149474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721147785,"port_rcv_data":1235115312352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.428","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.439","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.08,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.451","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.461","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.472","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.481","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.491","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.500","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.3079007,"monotonic":2349968.774405868,"hcas":{"mlx5_0":{"port_xmit_data":294439053994399,"port_rcv_data":294784164149474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721147785,"port_rcv_data":1235115312352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.407","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.419","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.431","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.442","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.452","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.461","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.471","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.480","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.3087368,"monotonic":2349969.775241641,"hcas":{"mlx5_0":{"port_xmit_data":294439053994399,"port_rcv_data":294784164149474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721147785,"port_rcv_data":1235115312352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.432","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.446","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.45,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.456","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.466","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.476","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.485","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.494","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.502","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.3095229,"monotonic":2349970.776027498,"hcas":{"mlx5_0":{"port_xmit_data":294439053994399,"port_rcv_data":294784164149474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721147785,"port_rcv_data":1235115312352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.455","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.4,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.478","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.489","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.498","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.508","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.517","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.526","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.3102937,"monotonic":2349971.776798371,"hcas":{"mlx5_0":{"port_xmit_data":294439053994399,"port_rcv_data":294784164149474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721147785,"port_rcv_data":1235115312352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.449","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.462","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.473","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.485","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.495","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.505","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.525","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.311061,"monotonic":2349972.777566175,"hcas":{"mlx5_0":{"port_xmit_data":294439053994399,"port_rcv_data":294784164149474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721147785,"port_rcv_data":1235115312352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.412","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.424","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.435","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.446","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.456","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.466","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.475","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.484","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.3118846,"monotonic":2349973.778389001,"hcas":{"mlx5_0":{"port_xmit_data":294439054166810,"port_rcv_data":294784223820068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822721323531,"port_rcv_data":1235174985769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.411","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":125.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.425","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":119.61,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.437","gpu_util_pct":99.0,"memory_used_mib":65997.0,"power_w":111.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.448","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":115.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.460","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.469","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.478","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.487","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.3126922,"monotonic":2349974.779197583,"hcas":{"mlx5_0":{"port_xmit_data":294439169210331,"port_rcv_data":294784281733763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822836364254,"port_rcv_data":1235232866728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536756465852,"port_rcv_data":294749705917036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892754531696,"port_rcv_data":1193285408307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.435","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":276.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.447","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":298.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.458","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":293.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.469","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":307.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.480","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":162.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.490","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":152.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.499","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":164.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.508","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":164.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.3134904,"monotonic":2349975.779995662,"hcas":{"mlx5_0":{"port_xmit_data":294439169210331,"port_rcv_data":294784281733763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822836364254,"port_rcv_data":1235232866728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294536960846490,"port_rcv_data":294749706284970,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1892958928681,"port_rcv_data":1193285792744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.415","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":134.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.427","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":190.3,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.438","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":165.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.448","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":165.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.458","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":301.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.468","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":288.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.478","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":314.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.487","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":296.33,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.3143084,"monotonic":2349976.780813602,"hcas":{"mlx5_0":{"port_xmit_data":294439169210331,"port_rcv_data":294784281733763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822836364254,"port_rcv_data":1235232866728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537047506678,"port_rcv_data":294749706502958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893045567403,"port_rcv_data":1193286016877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.417","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":106.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.429","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":107.51,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.440","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.450","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":91.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.460","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":105.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.470","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":110.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.479","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":112.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.489","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":101.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.3151262,"monotonic":2349977.781631654,"hcas":{"mlx5_0":{"port_xmit_data":294439169210331,"port_rcv_data":294784281733763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822836364254,"port_rcv_data":1235232866728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537047506678,"port_rcv_data":294749706502958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893045567403,"port_rcv_data":1193286016877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.417","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.429","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.440","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.451","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.461","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.471","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.481","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.490","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.315942,"monotonic":2349978.782447441,"hcas":{"mlx5_0":{"port_xmit_data":294439169210331,"port_rcv_data":294784281733763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822836364254,"port_rcv_data":1235232866728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537047506678,"port_rcv_data":294749706502958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893045567403,"port_rcv_data":1193286016877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.417","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.429","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.440","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.451","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.461","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.470","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.480","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.489","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.316278,"monotonic":2349979.78278348,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.439","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.451","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.7,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.463","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.473","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.484","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.493","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.503","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.512","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.3170736,"monotonic":2349980.783578493,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.420","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.432","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.87,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.443","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.453","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.463","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.472","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.481","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.490","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.3178928,"monotonic":2349981.784397655,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.444","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.452","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.460","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.475","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.483","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.490","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.498","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.3187144,"monotonic":2349982.78521949,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.465","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.476","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.4,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.487","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.498","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.508","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.517","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.526","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.535","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.3194919,"monotonic":2349983.785996913,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.452","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.463","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.62,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.474","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.485","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.494","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.504","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.513","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.522","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.320287,"monotonic":2349984.786792217,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.489","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.509","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.518","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.527","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.536","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.3210647,"monotonic":2349985.787570123,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.464","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.475","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.485","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.496","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.505","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.514","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.524","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.3218505,"monotonic":2349986.788355288,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.468","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.490","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.3226283,"monotonic":2349987.789133762,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.468","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.49,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.489","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.3234124,"monotonic":2349988.789917891,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.490","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:52.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:52.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:52.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.3241968,"monotonic":2349989.790702121,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.448","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.460","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.470","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.481","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.490","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.499","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.508","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.517","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.3249984,"monotonic":2349990.791503122,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.490","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.520","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.529","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:54.538","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.325522,"monotonic":2349991.792026941,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.463","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.475","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.486","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.496","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.506","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.524","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.533","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.3263063,"monotonic":2349992.79281129,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.465","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.477","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.489","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.499","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.529","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.538","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.327091,"monotonic":2349993.793596221,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.407","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.418","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.429","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.439","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.449","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.458","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.467","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.476","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.3279374,"monotonic":2349994.794442544,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.472","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.484","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.494","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.505","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.514","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.523","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.541","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.328719,"monotonic":2349995.795223905,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.476","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.487","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.498","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.508","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.518","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.527","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.536","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.545","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.3295345,"monotonic":2349996.796039576,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.25,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.490","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.57,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.3303256,"monotonic":2349997.796830583,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.468","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.490","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.509","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.3311124,"monotonic":2349998.797617654,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.472","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.483","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.494","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.504","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.514","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.523","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.541","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.3318996,"monotonic":2349999.798404948,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.471","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.483","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.49,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.494","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.504","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.516","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.527","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.546","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.3326776,"monotonic":2350000.799182844,"hcas":{"mlx5_0":{"port_xmit_data":294439287132705,"port_rcv_data":294784281859520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954285806,"port_rcv_data":1235232992696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.457","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.468","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.480","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.490","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.500","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.520","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.529","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.45,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.3334777,"monotonic":2350001.799982757,"hcas":{"mlx5_0":{"port_xmit_data":294439287314530,"port_rcv_data":294784341539667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1822954474066,"port_rcv_data":1235292679201,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.434","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":121.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.446","gpu_util_pct":85.0,"memory_used_mib":65879.0,"power_w":127.06,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.458","gpu_util_pct":35.0,"memory_used_mib":65997.0,"power_w":104.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.470","gpu_util_pct":81.0,"memory_used_mib":65879.0,"power_w":137.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.481","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.490","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.500","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.509","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.334305,"monotonic":2350002.800809508,"hcas":{"mlx5_0":{"port_xmit_data":294439402368753,"port_rcv_data":294784401324775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823069528416,"port_rcv_data":1235352465165,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537166257021,"port_rcv_data":294749706631671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893164317027,"port_rcv_data":1193286143001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.435","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":269.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.446","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":283.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.458","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":277.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.469","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":276.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.479","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":179.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.489","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":172.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.499","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":178.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.510","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":173.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.3351147,"monotonic":2350003.801619647,"hcas":{"mlx5_0":{"port_xmit_data":294439517309706,"port_rcv_data":294784425968726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823184472662,"port_rcv_data":1235377109227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537370651845,"port_rcv_data":294749707013793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893368713771,"port_rcv_data":1193286526809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.435","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":299.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.446","gpu_util_pct":36.0,"memory_used_mib":65879.0,"power_w":320.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.457","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":289.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.467","gpu_util_pct":97.0,"memory_used_mib":65879.0,"power_w":288.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.477","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":300.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.487","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":292.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.497","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":292.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:07.506","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":300.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.3359268,"monotonic":2350004.802431908,"hcas":{"mlx5_0":{"port_xmit_data":294439517309706,"port_rcv_data":294784425968726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823184472662,"port_rcv_data":1235377109227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537575004974,"port_rcv_data":294749707358264,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893573059185,"port_rcv_data":1193286862357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.441","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.452","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":86.58,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.463","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.474","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.484","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":270.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.493","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":241.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.503","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":246.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.512","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":244.5,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.3367434,"monotonic":2350005.803248896,"hcas":{"mlx5_0":{"port_xmit_data":294439517309706,"port_rcv_data":294784425968726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823184472662,"port_rcv_data":1235377109227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537611807180,"port_rcv_data":294749707458700,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893609860200,"port_rcv_data":1193286965169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.440","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.451","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.44,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.462","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.92,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.473","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.483","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.492","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.502","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.511","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.3375611,"monotonic":2350006.804066504,"hcas":{"mlx5_0":{"port_xmit_data":294439517309706,"port_rcv_data":294784425968726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823184472662,"port_rcv_data":1235377109227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537611807180,"port_rcv_data":294749707458700,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893609860200,"port_rcv_data":1193286965169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.440","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.451","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.462","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.473","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.483","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.492","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.502","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.511","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.3383796,"monotonic":2350007.8048849,"hcas":{"mlx5_0":{"port_xmit_data":294439517309706,"port_rcv_data":294784425968726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823184472662,"port_rcv_data":1235377109227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537611807180,"port_rcv_data":294749707458700,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893609860200,"port_rcv_data":1193286965169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.439","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.450","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.461","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.471","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.481","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.491","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.500","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.509","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.3392067,"monotonic":2350008.805711844,"hcas":{"mlx5_0":{"port_xmit_data":294439571511566,"port_rcv_data":294784426032913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823238673110,"port_rcv_data":1235377172514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537666836989,"port_rcv_data":294749707525167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893664888720,"port_rcv_data":1193287029292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.458","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.470","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.39,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.481","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.492","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.502","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.512","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.522","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.531","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.3400044,"monotonic":2350009.806508651,"hcas":{"mlx5_0":{"port_xmit_data":294439571511566,"port_rcv_data":294784426032913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823238673110,"port_rcv_data":1235377172514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537666836989,"port_rcv_data":294749707525167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893664888720,"port_rcv_data":1193287029292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.444","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.456","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.32,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.468","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.489","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.499","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.509","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.518","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.3408318,"monotonic":2350010.807336568,"hcas":{"mlx5_0":{"port_xmit_data":294439577745792,"port_rcv_data":294784485705213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823253195952,"port_rcv_data":1235436847513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537666836989,"port_rcv_data":294749707525167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893664888720,"port_rcv_data":1193287029292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.470","gpu_util_pct":46.0,"memory_used_mib":65879.0,"power_w":190.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.482","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":216.85,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.493","gpu_util_pct":77.0,"memory_used_mib":65997.0,"power_w":203.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.504","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":193.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.515","gpu_util_pct":93.0,"memory_used_mib":70475.0,"power_w":88.28,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.526","gpu_util_pct":95.0,"memory_used_mib":70475.0,"power_w":79.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.537","gpu_util_pct":92.0,"memory_used_mib":70475.0,"power_w":89.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.547","gpu_util_pct":1.0,"memory_used_mib":70475.0,"power_w":101.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.3415256,"monotonic":2350011.808030577,"hcas":{"mlx5_0":{"port_xmit_data":294439686716217,"port_rcv_data":294784543837010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823353875959,"port_rcv_data":1235494961143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537871210802,"port_rcv_data":294749707886354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893869269975,"port_rcv_data":1193287397472,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.459","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":271.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.471","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":274.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.481","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":263.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.492","gpu_util_pct":28.0,"memory_used_mib":65879.0,"power_w":253.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.501","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":305.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.512","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":282.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.523","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":305.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.532","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":309.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.3423295,"monotonic":2350012.80883458,"hcas":{"mlx5_0":{"port_xmit_data":294439686716217,"port_rcv_data":294784543837010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823353875959,"port_rcv_data":1235494961143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537958197310,"port_rcv_data":294749708073886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893956232966,"port_rcv_data":1193287582623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.465","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":104.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.476","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":103.19,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.487","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":96.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.497","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":93.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.506","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":237.84,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.516","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":224.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.525","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":219.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.534","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":219.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.3431337,"monotonic":2350013.80963886,"hcas":{"mlx5_0":{"port_xmit_data":294439686716217,"port_rcv_data":294784543837010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823353875959,"port_rcv_data":1235494961143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537958197310,"port_rcv_data":294749708073886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893956232966,"port_rcv_data":1193287582623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.443","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.454","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.465","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.475","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.485","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":107.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.494","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":86.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.503","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":101.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.512","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":107.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.3439596,"monotonic":2350014.810464745,"hcas":{"mlx5_0":{"port_xmit_data":294439686716217,"port_rcv_data":294784543837010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823353875959,"port_rcv_data":1235494961143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537958197310,"port_rcv_data":294749708073886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893956232966,"port_rcv_data":1193287582623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.447","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.458","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.468","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.478","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.488","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.497","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.506","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.3447795,"monotonic":2350015.811284804,"hcas":{"mlx5_0":{"port_xmit_data":294439686716217,"port_rcv_data":294784543837010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823353875959,"port_rcv_data":1235494961143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294537958197310,"port_rcv_data":294749708073886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1893956232966,"port_rcv_data":1193287582623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.447","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.458","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.4,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.469","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.480","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.490","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.499","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.508","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.517","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.3449512,"monotonic":2350016.811455809,"hcas":{"mlx5_0":{"port_xmit_data":294439805536015,"port_rcv_data":294784543963354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823472694967,"port_rcv_data":1235495086724,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294538077845011,"port_rcv_data":294749708203153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1894075880046,"port_rcv_data":1193287712544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.462","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.474","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.35,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.485","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.496","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.506","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.516","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.526","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.535","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.3457484,"monotonic":2350017.812253504,"hcas":{"mlx5_0":{"port_xmit_data":294439805722361,"port_rcv_data":294784603647774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823472881615,"port_rcv_data":1235554771307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294538077845011,"port_rcv_data":294749708203153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1894075880046,"port_rcv_data":1193287712544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.451","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":197.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.464","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":216.96,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.475","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":197.9,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.486","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":216.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.496","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.506","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:21.524","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.346572,"monotonic":2350018.813076873,"hcas":{"mlx5_0":{"port_xmit_data":294439920772947,"port_rcv_data":294784663430176,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823587928462,"port_rcv_data":1235614549334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294538077845011,"port_rcv_data":294749708203153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1894075880046,"port_rcv_data":1193287712544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.447","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":271.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.460","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":299.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.472","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":267.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.483","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":269.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.493","gpu_util_pct":20.0,"memory_used_mib":70475.0,"power_w":206.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.503","gpu_util_pct":52.0,"memory_used_mib":70475.0,"power_w":181.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.513","gpu_util_pct":27.0,"memory_used_mib":70475.0,"power_w":172.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.523","gpu_util_pct":86.0,"memory_used_mib":70475.0,"power_w":160.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.3473852,"monotonic":2350019.813890238,"hcas":{"mlx5_0":{"port_xmit_data":294440035716937,"port_rcv_data":294784687613830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823702874043,"port_rcv_data":1235638732950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294538282194535,"port_rcv_data":294749708540563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1894280257126,"port_rcv_data":1193288076967,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.486","gpu_util_pct":26.0,"memory_used_mib":65879.0,"power_w":226.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.500","gpu_util_pct":32.0,"memory_used_mib":65879.0,"power_w":226.71,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.511","gpu_util_pct":57.0,"memory_used_mib":65997.0,"power_w":206.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.523","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":230.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.533","gpu_util_pct":96.0,"memory_used_mib":70475.0,"power_w":298.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:23.545","gpu_util_pct":95.0,"memory_used_mib":70475.0,"power_w":327.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:23.556","gpu_util_pct":96.0,"memory_used_mib":70475.0,"power_w":318.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:23.567","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":305.38,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.3481545,"monotonic":2350020.814659811,"hcas":{"mlx5_0":{"port_xmit_data":294440150912292,"port_rcv_data":294784805708352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823818094530,"port_rcv_data":1235756822059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294538522669711,"port_rcv_data":294749709001014,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1894520725968,"port_rcv_data":1193288536544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.459","gpu_util_pct":98.0,"memory_used_mib":65879.0,"power_w":245.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.471","gpu_util_pct":99.0,"memory_used_mib":65879.0,"power_w":249.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.483","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":269.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.494","gpu_util_pct":85.0,"memory_used_mib":65879.0,"power_w":281.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.504","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":241.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.514","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":220.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.524","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":213.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.533","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":217.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.3489575,"monotonic":2350021.815462757,"hcas":{"mlx5_0":{"port_xmit_data":294440151078594,"port_rcv_data":294784865371596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823818261367,"port_rcv_data":1235816485869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294538727027909,"port_rcv_data":294749709347401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1894725100198,"port_rcv_data":1193288898914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.453","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":307.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.466","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":322.11,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.478","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":308.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.489","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":309.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:25.500","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":274.23,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:25.510","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":287.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:25.521","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":257.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:25.531","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":270.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.3495238,"monotonic":2350022.816028712,"hcas":{"mlx5_0":{"port_xmit_data":294440266107309,"port_rcv_data":294784919383460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823933283380,"port_rcv_data":1235870489204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294538814501046,"port_rcv_data":294749709577615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1894812540694,"port_rcv_data":1193289127572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.479","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":268.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.492","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":275.39,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.503","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":244.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.515","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":247.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.526","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":319.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.536","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":311.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.547","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":325.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.558","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":325.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.3503022,"monotonic":2350023.816807592,"hcas":{"mlx5_0":{"port_xmit_data":294440266107509,"port_rcv_data":294784919383608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823933283380,"port_rcv_data":1235870489204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539018876420,"port_rcv_data":294749709939914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895016938393,"port_rcv_data":1193289514366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.455","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":183.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.466","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":177.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.477","gpu_util_pct":67.0,"memory_used_mib":65997.0,"power_w":157.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.488","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":163.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.498","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":212.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.510","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":207.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.520","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":198.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.531","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":223.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.35111,"monotonic":2350024.817615272,"hcas":{"mlx5_0":{"port_xmit_data":294440266108099,"port_rcv_data":294784919384146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823933283380,"port_rcv_data":1235870489204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539099696116,"port_rcv_data":294749710134947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895097764293,"port_rcv_data":1193289719187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.469","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.480","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.491","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.502","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.512","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":207.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.521","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":211.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.531","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":204.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.540","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":193.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.351912,"monotonic":2350025.818417252,"hcas":{"mlx5_0":{"port_xmit_data":294440266108099,"port_rcv_data":294784919384146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823933283380,"port_rcv_data":1235870489204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539099696654,"port_rcv_data":294749710135537,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895097764293,"port_rcv_data":1193289719187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.451","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.462","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.22,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.473","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.484","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.494","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.504","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.513","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.522","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.3525817,"monotonic":2350026.81908607,"hcas":{"mlx5_0":{"port_xmit_data":294440319550407,"port_rcv_data":294784979086475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1823986701421,"port_rcv_data":1235930167844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539099701547,"port_rcv_data":294749710137042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895097768973,"port_rcv_data":1193289720427,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.457","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":187.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.469","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":192.74,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.482","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":194.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.495","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":187.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.505","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.524","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.533","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.3534048,"monotonic":2350027.819909452,"hcas":{"mlx5_0":{"port_xmit_data":294440434607330,"port_rcv_data":294785036471699,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824101748524,"port_rcv_data":1235987519379,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539153827931,"port_rcv_data":294749710199941,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895151895357,"port_rcv_data":1193289784737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.456","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":220.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.468","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":241.29,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.480","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":235.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.491","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":226.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.502","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":186.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.513","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":182.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.523","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":194.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.533","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":209.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.3542144,"monotonic":2350028.820719224,"hcas":{"mlx5_0":{"port_xmit_data":294440554320486,"port_rcv_data":294785036599844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824221460500,"port_rcv_data":1235987644262,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539358202933,"port_rcv_data":294749710559553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895356274012,"port_rcv_data":1193290147999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.454","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":180.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.465","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":176.27,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.476","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":196.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.486","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":182.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.496","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":270.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.506","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":277.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.516","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":252.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.526","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":233.45,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.3550365,"monotonic":2350029.821541483,"hcas":{"mlx5_0":{"port_xmit_data":294440646054441,"port_rcv_data":294785036697634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824295842924,"port_rcv_data":1235987725591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539564593402,"port_rcv_data":294749710901251,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895562647077,"port_rcv_data":1193290499577,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.452","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.5,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.463","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.58,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.474","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.484","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.493","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":172.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.503","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":162.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.514","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":156.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:33.523","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":149.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.3558671,"monotonic":2350030.822372107,"hcas":{"mlx5_0":{"port_xmit_data":294440665254682,"port_rcv_data":294785096405677,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824332388802,"port_rcv_data":1236047444334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539639793045,"port_rcv_data":294749710983110,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895654799647,"port_rcv_data":1193290596886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.457","gpu_util_pct":20.0,"memory_used_mib":65879.0,"power_w":213.23,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.468","gpu_util_pct":26.0,"memory_used_mib":65879.0,"power_w":217.19,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.479","gpu_util_pct":19.0,"memory_used_mib":65997.0,"power_w":200.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.491","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":212.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.501","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.511","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.525","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.536","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.3566782,"monotonic":2350031.823182201,"hcas":{"mlx5_0":{"port_xmit_data":294440780290729,"port_rcv_data":294785155773248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824447422783,"port_rcv_data":1236106777512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539676161584,"port_rcv_data":294749711021243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895674213905,"port_rcv_data":1193290618121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.483","gpu_util_pct":96.0,"memory_used_mib":65879.0,"power_w":282.98,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.495","gpu_util_pct":96.0,"memory_used_mib":65879.0,"power_w":287.89,"sm_clock_mhz":2415.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.507","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":260.37,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.518","gpu_util_pct":96.0,"memory_used_mib":65879.0,"power_w":272.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.528","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":236.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.538","gpu_util_pct":75.0,"memory_used_mib":70475.0,"power_w":226.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.547","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":237.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.555","gpu_util_pct":41.0,"memory_used_mib":70475.0,"power_w":227.16,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.357465,"monotonic":2350032.823970262,"hcas":{"mlx5_0":{"port_xmit_data":294440780290729,"port_rcv_data":294785155773248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824447422783,"port_rcv_data":1236106777512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539880518854,"port_rcv_data":294749711365844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895878571077,"port_rcv_data":1193290963008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.493","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.505","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.92,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.516","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":83.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.526","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.536","gpu_util_pct":96.0,"memory_used_mib":70475.0,"power_w":287.7,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.546","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":294.09,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.555","gpu_util_pct":98.0,"memory_used_mib":70475.0,"power_w":276.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.565","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":294.38,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.3582404,"monotonic":2350033.824745307,"hcas":{"mlx5_0":{"port_xmit_data":294440780423750,"port_rcv_data":294785215403491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824447563168,"port_rcv_data":1236166415522,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539969351282,"port_rcv_data":294749711565004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895967369084,"port_rcv_data":1193291156803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.461","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":217.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.472","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":213.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.483","gpu_util_pct":38.0,"memory_used_mib":65997.0,"power_w":214.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.493","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":206.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.503","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":107.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.513","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":109.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.522","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":105.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.531","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":87.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.3590465,"monotonic":2350034.825551918,"hcas":{"mlx5_0":{"port_xmit_data":294440895437952,"port_rcv_data":294785274715704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824562583289,"port_rcv_data":1236225710792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294539969351282,"port_rcv_data":294749711565004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1895967369084,"port_rcv_data":1193291156803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.500","gpu_util_pct":99.0,"memory_used_mib":65879.0,"power_w":287.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.509","gpu_util_pct":94.0,"memory_used_mib":65879.0,"power_w":274.7,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.518","gpu_util_pct":93.0,"memory_used_mib":65997.0,"power_w":253.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.527","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":269.05,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":245.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.538","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":237.59,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.544","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":240.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.549","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":223.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.3598835,"monotonic":2350035.82638817,"hcas":{"mlx5_0":{"port_xmit_data":294441012462305,"port_rcv_data":294785274844007,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824679607385,"port_rcv_data":1236225836559,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540291581798,"port_rcv_data":294749712057844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896289608097,"port_rcv_data":1193291659204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.465","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":104.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.477","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.488","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":85.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.499","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":86.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.509","gpu_util_pct":79.0,"memory_used_mib":70475.0,"power_w":283.81,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.519","gpu_util_pct":87.0,"memory_used_mib":70475.0,"power_w":275.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.529","gpu_util_pct":85.0,"memory_used_mib":70475.0,"power_w":289.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:39.538","gpu_util_pct":90.0,"memory_used_mib":70475.0,"power_w":286.24,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.360687,"monotonic":2350036.827192317,"hcas":{"mlx5_0":{"port_xmit_data":294441012462605,"port_rcv_data":294785274844085,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824679607385,"port_rcv_data":1236225836559,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540380359071,"port_rcv_data":294749712252043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896378363523,"port_rcv_data":1193291848776,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.464","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.475","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.84,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.487","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.497","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.507","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.517","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.527","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":86.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.536","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.3615115,"monotonic":2350037.828016473,"hcas":{"mlx5_0":{"port_xmit_data":294441012462805,"port_rcv_data":294785274844233,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824679607385,"port_rcv_data":1236225836559,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540380359071,"port_rcv_data":294749712252043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896378363523,"port_rcv_data":1193291848776,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.465","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.477","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.4,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.488","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.499","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.509","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.538","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.362344,"monotonic":2350038.828848812,"hcas":{"mlx5_0":{"port_xmit_data":294441133974121,"port_rcv_data":294785274973712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824801118176,"port_rcv_data":1236225966635,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540492401732,"port_rcv_data":294749712374758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896495242050,"port_rcv_data":1193291973334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.465","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.477","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.67,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.488","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.498","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.508","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.518","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:42.527","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:42.536","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.3631635,"monotonic":2350039.829668129,"hcas":{"mlx5_0":{"port_xmit_data":294441133974706,"port_rcv_data":294785274973867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824801118761,"port_rcv_data":1236225966790,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540502698732,"port_rcv_data":294749712385484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896500702971,"port_rcv_data":1193291979643,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.505","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.517","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.32,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.527","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.538","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.548","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.557","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.566","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.575","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.3639421,"monotonic":2350040.83044736,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.445","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.456","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.78,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.467","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.477","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.487","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.497","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.506","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.3647816,"monotonic":2350041.831286853,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.489","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.511","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.522","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:45.541","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:45.551","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:45.560","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.3655758,"monotonic":2350042.832081059,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.488","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.499","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.4,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.510","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.520","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.530","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.540","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.549","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.558","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.3663719,"monotonic":2350043.832877148,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.445","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.456","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.467","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.478","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.487","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.497","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.506","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.3672128,"monotonic":2350044.833718273,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.507","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.519","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.530","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.540","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.550","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.559","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.569","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.577","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.3679914,"monotonic":2350045.834497075,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.478","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.69,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.489","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.538","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.3688104,"monotonic":2350046.835316057,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.492","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.503","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.87,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.514","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.524","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.534","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.543","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:50.552","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:50.561","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.3696077,"monotonic":2350047.836113118,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.516","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.78,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.527","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.538","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:51.548","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:51.558","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:51.568","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.577","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.586","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.3703804,"monotonic":2350048.836885739,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.523","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.534","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.544","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.554","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.564","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.573","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.583","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.3711565,"monotonic":2350049.837661946,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.462","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.473","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.49,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.484","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.494","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.504","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:53.513","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:53.522","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.531","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.3719854,"monotonic":2350050.838490832,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.523","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.59,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.534","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.544","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.554","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.563","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.572","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.581","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.3727655,"monotonic":2350051.839270961,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.516","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.527","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.538","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.548","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.558","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.568","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.577","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.586","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.373532,"monotonic":2350052.840037365,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.471","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.483","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.494","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:56.504","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:56.514","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:56.524","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:56.533","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:56.542","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.374357,"monotonic":2350053.840862248,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.523","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.534","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.544","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.554","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.563","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.573","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.581","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.3751407,"monotonic":2350054.841646091,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.516","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.527","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.538","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.549","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.558","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:58.568","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.578","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.586","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.3759239,"monotonic":2350055.842429005,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.520","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.531","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.74,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.542","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.552","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.562","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.571","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.580","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.589","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.3767092,"monotonic":2350056.843213996,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.516","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.527","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.538","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:00.548","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:00.558","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:00.567","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:00.576","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:00.585","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.3774912,"monotonic":2350057.843996726,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.523","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.535","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.546","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.556","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.566","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.575","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.584","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.593","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.3782759,"monotonic":2350058.844780689,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.520","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.531","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.542","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.552","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:02.562","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:02.572","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.581","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.590","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.3790565,"monotonic":2350059.84556151,"hcas":{"mlx5_0":{"port_xmit_data":294441255486807,"port_rcv_data":294785275104607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922630137,"port_rcv_data":1236226096616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.458","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.469","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.480","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.490","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.500","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.520","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.529","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.3798974,"monotonic":2350060.846402745,"hcas":{"mlx5_0":{"port_xmit_data":294441255671374,"port_rcv_data":294785334787061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1824922826411,"port_rcv_data":1236285790949,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540625039469,"port_rcv_data":294749712517889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896623042419,"port_rcv_data":1193292109982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.481","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":169.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.492","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":174.45,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.503","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":150.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.513","gpu_util_pct":50.0,"memory_used_mib":65879.0,"power_w":164.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.524","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.534","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.541","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.548","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.3807182,"monotonic":2350061.84722364,"hcas":{"mlx5_0":{"port_xmit_data":294441370704673,"port_rcv_data":294785393519539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825037854133,"port_rcv_data":1236344498934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540829432034,"port_rcv_data":294749712898061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896827441971,"port_rcv_data":1193292496708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.498","gpu_util_pct":40.0,"memory_used_mib":65879.0,"power_w":257.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.510","gpu_util_pct":35.0,"memory_used_mib":65879.0,"power_w":272.9,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.521","gpu_util_pct":8.0,"memory_used_mib":65997.0,"power_w":266.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.531","gpu_util_pct":92.0,"memory_used_mib":65879.0,"power_w":252.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.541","gpu_util_pct":98.0,"memory_used_mib":70475.0,"power_w":203.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.551","gpu_util_pct":98.0,"memory_used_mib":70475.0,"power_w":184.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.561","gpu_util_pct":98.0,"memory_used_mib":70475.0,"power_w":185.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.571","gpu_util_pct":67.0,"memory_used_mib":70475.0,"power_w":185.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.3815196,"monotonic":2350062.848024966,"hcas":{"mlx5_0":{"port_xmit_data":294441370704673,"port_rcv_data":294785393519539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825037854133,"port_rcv_data":1236344498934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540917312877,"port_rcv_data":294749713092260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896915314260,"port_rcv_data":1193292693575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":96.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.491","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":95.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.502","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":82.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:06.523","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":271.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:06.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":279.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.542","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":269.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.551","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":242.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.382341,"monotonic":2350063.848846254,"hcas":{"mlx5_0":{"port_xmit_data":294441370704673,"port_rcv_data":294785393519539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825037854133,"port_rcv_data":1236344498934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540917312877,"port_rcv_data":294749713092260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896915314260,"port_rcv_data":1193292693575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.461","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.472","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.483","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.493","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.502","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.512","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":102.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.521","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":113.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.530","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":89.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.3831813,"monotonic":2350064.849686869,"hcas":{"mlx5_0":{"port_xmit_data":294441370704673,"port_rcv_data":294785393519539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825037854133,"port_rcv_data":1236344498934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540917312877,"port_rcv_data":294749713092260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896915314260,"port_rcv_data":1193292693575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.483","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.494","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.23,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.505","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.515","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.525","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.534","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.544","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.553","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.3840034,"monotonic":2350065.850508732,"hcas":{"mlx5_0":{"port_xmit_data":294441370704673,"port_rcv_data":294785393519539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825037854133,"port_rcv_data":1236344498934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540917312877,"port_rcv_data":294749713092260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896915314260,"port_rcv_data":1193292693575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.463","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.474","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.485","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.495","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.505","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.515","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.524","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.533","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259210.3848574,"monotonic":2350066.851362312,"hcas":{"mlx5_0":{"port_xmit_data":294441490417247,"port_rcv_data":294785393645635,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825157565917,"port_rcv_data":1236344625646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294540917317557,"port_rcv_data":294749713093500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1896915318940,"port_rcv_data":1193292694815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.31,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.523","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.535","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.545","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.556","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.566","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.576","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.3856575,"monotonic":2350067.852162802,"hcas":{"mlx5_0":{"port_xmit_data":294441490601233,"port_rcv_data":294785453324503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825157757345,"port_rcv_data":1236404311956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541037858132,"port_rcv_data":294749713224213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897035858796,"port_rcv_data":1193292823697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.487","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":142.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.501","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":170.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.513","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":152.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.524","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":163.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.534","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:11.544","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.553","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.562","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.38647,"monotonic":2350068.852975429,"hcas":{"mlx5_0":{"port_xmit_data":294441605627748,"port_rcv_data":294785510183094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825272785880,"port_rcv_data":1236461126153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541037858132,"port_rcv_data":294749713224213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897035858796,"port_rcv_data":1193292823697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.488","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":285.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.501","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":308.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.513","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":304.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.523","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":283.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.534","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":219.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.544","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":230.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.553","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":217.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.562","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":216.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.387282,"monotonic":2350069.853787056,"hcas":{"mlx5_0":{"port_xmit_data":294441605627748,"port_rcv_data":294785510183094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825272785880,"port_rcv_data":1236461126153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541242233114,"port_rcv_data":294749713587004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897240242376,"port_rcv_data":1193293194341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.469","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":158.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.480","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":178.86,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.492","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":180.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.502","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":140.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.511","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":297.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:13.521","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":305.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.530","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":290.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.539","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":300.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.3881192,"monotonic":2350070.854624558,"hcas":{"mlx5_0":{"port_xmit_data":294441605627748,"port_rcv_data":294785510183094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825272785880,"port_rcv_data":1236461126153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541327301087,"port_rcv_data":294749713771811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897325264669,"port_rcv_data":1193293378198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.467","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":98.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":100.88,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.489","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":83.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":101.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":105.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":134.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":109.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.3889618,"monotonic":2350071.855467047,"hcas":{"mlx5_0":{"port_xmit_data":294441605627748,"port_rcv_data":294785510183094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825272785880,"port_rcv_data":1236461126153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541327301087,"port_rcv_data":294749713771811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897325264669,"port_rcv_data":1193293378198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.491","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.502","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:15.513","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:15.524","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:15.534","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":99.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.543","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.553","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":110.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.562","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.3897822,"monotonic":2350072.856287572,"hcas":{"mlx5_0":{"port_xmit_data":294441605627748,"port_rcv_data":294785510183094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825272785880,"port_rcv_data":1236461126153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541327301087,"port_rcv_data":294749713771811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897325264669,"port_rcv_data":1193293378198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.468","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.479","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.490","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.510","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.519","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.528","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.537","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.3906288,"monotonic":2350073.857134236,"hcas":{"mlx5_0":{"port_xmit_data":294441605770584,"port_rcv_data":294785569823151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825272926984,"port_rcv_data":1236520764943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541327301087,"port_rcv_data":294749713771811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897325264669,"port_rcv_data":1193293378198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.490","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":125.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.502","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":148.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.513","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":128.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.525","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":132.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.535","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.544","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.553","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.562","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.3914552,"monotonic":2350074.857960126,"hcas":{"mlx5_0":{"port_xmit_data":294441720798872,"port_rcv_data":294785628643534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825387959376,"port_rcv_data":1236579548282,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541327301087,"port_rcv_data":294749713771811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897325264669,"port_rcv_data":1193293378198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.492","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":261.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.504","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":275.27,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.515","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":267.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.526","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":277.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.536","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":157.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.546","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":233.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.555","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":156.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.564","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":150.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.3922715,"monotonic":2350075.858777015,"hcas":{"mlx5_0":{"port_xmit_data":294441837105537,"port_rcv_data":294785688447365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825504258650,"port_rcv_data":1236639345233,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541531684940,"port_rcv_data":294749714140063,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897529662002,"port_rcv_data":1193293760376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.473","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":278.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.485","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":308.52,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.496","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":268.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.506","gpu_util_pct":12.0,"memory_used_mib":65879.0,"power_w":265.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.516","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":310.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.527","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":288.74,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.537","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":301.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.547","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":282.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.3930995,"monotonic":2350076.859604888,"hcas":{"mlx5_0":{"port_xmit_data":294441952144177,"port_rcv_data":294785747481301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825619302998,"port_rcv_data":1236698380420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294541736695936,"port_rcv_data":294749714505162,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1897734641794,"port_rcv_data":1193294131173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.509","gpu_util_pct":25.0,"memory_used_mib":65879.0,"power_w":255.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.518","gpu_util_pct":19.0,"memory_used_mib":65879.0,"power_w":290.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.527","gpu_util_pct":25.0,"memory_used_mib":65997.0,"power_w":286.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.536","gpu_util_pct":80.0,"memory_used_mib":65879.0,"power_w":275.33,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:20.545","gpu_util_pct":17.0,"memory_used_mib":70475.0,"power_w":280.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.554","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":256.99,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.562","gpu_util_pct":11.0,"memory_used_mib":70475.0,"power_w":277.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.570","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":249.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.3939161,"monotonic":2350077.860421727,"hcas":{"mlx5_0":{"port_xmit_data":294441952144177,"port_rcv_data":294785747481301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825619302998,"port_rcv_data":1236698380420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542029384114,"port_rcv_data":294749715042129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898027318862,"port_rcv_data":1193294664332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.473","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.484","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.78,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.495","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":100.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.506","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.516","gpu_util_pct":3.0,"memory_used_mib":70475.0,"power_w":287.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.526","gpu_util_pct":35.0,"memory_used_mib":70475.0,"power_w":275.08,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.535","gpu_util_pct":10.0,"memory_used_mib":70475.0,"power_w":279.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.544","gpu_util_pct":44.0,"memory_used_mib":70475.0,"power_w":275.76,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.3947604,"monotonic":2350078.861265902,"hcas":{"mlx5_0":{"port_xmit_data":294441952144177,"port_rcv_data":294785747481301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825619302998,"port_rcv_data":1236698380420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542029384114,"port_rcv_data":294749715042129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898027318862,"port_rcv_data":1193294664332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.492","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.504","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.514","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.524","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:22.534","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.543","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.552","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":85.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.561","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":102.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.3955867,"monotonic":2350079.862092275,"hcas":{"mlx5_0":{"port_xmit_data":294441952144377,"port_rcv_data":294785747481449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825619302998,"port_rcv_data":1236698380420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542029384114,"port_rcv_data":294749715042129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898027318862,"port_rcv_data":1193294664332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.495","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.506","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.517","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.528","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.538","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.547","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.557","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.566","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.3964062,"monotonic":2350080.862911766,"hcas":{"mlx5_0":{"port_xmit_data":294442071856751,"port_rcv_data":294785747608746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825739014782,"port_rcv_data":1236698509039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542029388794,"port_rcv_data":294749715043369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898027323542,"port_rcv_data":1193294665572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.513","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.524","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.535","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.545","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.555","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.565","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.574","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.583","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.3972127,"monotonic":2350081.863717778,"hcas":{"mlx5_0":{"port_xmit_data":294442071856751,"port_rcv_data":294785747608746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825739014782,"port_rcv_data":1236698509039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542029388794,"port_rcv_data":294749715043369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898027323542,"port_rcv_data":1193294665572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.496","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.508","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.09,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.519","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:25.530","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:25.540","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:25.549","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.559","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.568","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.398031,"monotonic":2350082.864536344,"hcas":{"mlx5_0":{"port_xmit_data":294442192476011,"port_rcv_data":294785747741737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825859633382,"port_rcv_data":1236698640333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542271372567,"port_rcv_data":294749715306831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898269305390,"port_rcv_data":1193294925340,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.499","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.511","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.76,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.522","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.532","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.542","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.551","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.560","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.570","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.3988621,"monotonic":2350083.865366932,"hcas":{"mlx5_0":{"port_xmit_data":294442192649360,"port_rcv_data":294785807413222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825859802564,"port_rcv_data":1236758307853,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542271372567,"port_rcv_data":294749715306831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898269305390,"port_rcv_data":1193294925340,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.478","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":210.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:27.489","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":221.13,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:27.500","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":197.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:27.510","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":205.8,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.520","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.529","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.538","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.547","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.3997154,"monotonic":2350084.866220168,"hcas":{"mlx5_0":{"port_xmit_data":294442307681066,"port_rcv_data":294785864884228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825974843501,"port_rcv_data":1236815780274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542422098567,"port_rcv_data":294749715564186,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898425308181,"port_rcv_data":1193295193307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.502","gpu_util_pct":96.0,"memory_used_mib":65879.0,"power_w":288.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.513","gpu_util_pct":96.0,"memory_used_mib":65879.0,"power_w":280.13,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.525","gpu_util_pct":96.0,"memory_used_mib":65997.0,"power_w":252.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.535","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":258.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.545","gpu_util_pct":31.0,"memory_used_mib":70475.0,"power_w":160.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.555","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":173.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.565","gpu_util_pct":24.0,"memory_used_mib":70475.0,"power_w":159.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.575","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":167.31,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.400532,"monotonic":2350085.867037033,"hcas":{"mlx5_0":{"port_xmit_data":294442307681066,"port_rcv_data":294785864884228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825974843501,"port_rcv_data":1236815780274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542561783769,"port_rcv_data":294749715901378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898559714634,"port_rcv_data":1193295530658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.480","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":98.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.491","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":93.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.502","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":84.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.522","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":289.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.531","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":272.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.541","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":268.38,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.550","gpu_util_pct":2.0,"memory_used_mib":70475.0,"power_w":259.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.4013774,"monotonic":2350086.867882873,"hcas":{"mlx5_0":{"port_xmit_data":294442307681066,"port_rcv_data":294785864884228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825974843501,"port_rcv_data":1236815780274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542561783769,"port_rcv_data":294749715901378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898559714634,"port_rcv_data":1193295530658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.481","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.492","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.36,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.502","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.513","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.523","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.533","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.542","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.551","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.402224,"monotonic":2350087.868729372,"hcas":{"mlx5_0":{"port_xmit_data":294442307681066,"port_rcv_data":294785864884228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825974843501,"port_rcv_data":1236815780274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542561783769,"port_rcv_data":294749715901378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898559714634,"port_rcv_data":1193295530658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.481","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.492","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.503","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.513","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.523","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.541","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.550","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.4030695,"monotonic":2350088.869574863,"hcas":{"mlx5_0":{"port_xmit_data":294442307681066,"port_rcv_data":294785864884228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1825974843501,"port_rcv_data":1236815780274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542561783769,"port_rcv_data":294749715901378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898559714634,"port_rcv_data":1193295530658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.482","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.493","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.504","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.514","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.523","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.533","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.542","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.551","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.4039161,"monotonic":2350089.870421187,"hcas":{"mlx5_0":{"port_xmit_data":294442425603375,"port_rcv_data":294785865009920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826092765053,"port_rcv_data":1236815905948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542680534176,"port_rcv_data":294749716028870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898678464258,"port_rcv_data":1193295658161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.523","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.534","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.03,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.545","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:33.556","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.566","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.575","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.585","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.594","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.4047196,"monotonic":2350090.87122467,"hcas":{"mlx5_0":{"port_xmit_data":294442425603375,"port_rcv_data":294785865009920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826092765053,"port_rcv_data":1236815905948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542680534176,"port_rcv_data":294749716028870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898678464258,"port_rcv_data":1193295658161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.507","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.518","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.69,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.529","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.539","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.549","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.559","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.568","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.577","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.4055436,"monotonic":2350091.872049089,"hcas":{"mlx5_0":{"port_xmit_data":294442425793257,"port_rcv_data":294785924698077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826092942426,"port_rcv_data":1236875580946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542680534176,"port_rcv_data":294749716028870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898678464258,"port_rcv_data":1193295658161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.503","gpu_util_pct":11.0,"memory_used_mib":65879.0,"power_w":208.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.515","gpu_util_pct":5.0,"memory_used_mib":65879.0,"power_w":211.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:35.526","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":189.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:35.536","gpu_util_pct":66.0,"memory_used_mib":65879.0,"power_w":196.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.547","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.557","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.566","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.573","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.406369,"monotonic":2350092.872874168,"hcas":{"mlx5_0":{"port_xmit_data":294442540831329,"port_rcv_data":294785983900544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826207980916,"port_rcv_data":1236934765368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542884902334,"port_rcv_data":294749716384107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898882848065,"port_rcv_data":1193296030257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.505","gpu_util_pct":68.0,"memory_used_mib":65879.0,"power_w":277.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.517","gpu_util_pct":62.0,"memory_used_mib":65879.0,"power_w":287.03,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.528","gpu_util_pct":39.0,"memory_used_mib":65997.0,"power_w":251.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.539","gpu_util_pct":96.0,"memory_used_mib":65879.0,"power_w":289.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.548","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":239.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.559","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":246.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.569","gpu_util_pct":91.0,"memory_used_mib":70475.0,"power_w":267.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.579","gpu_util_pct":33.0,"memory_used_mib":70475.0,"power_w":259.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.4071853,"monotonic":2350093.873690615,"hcas":{"mlx5_0":{"port_xmit_data":294442540831329,"port_rcv_data":294785983900544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826207980916,"port_rcv_data":1236934765368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542973492319,"port_rcv_data":294749716588139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898971422646,"port_rcv_data":1193296230389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.503","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":101.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.515","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":113.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:37.526","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":96.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:37.536","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":89.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.545","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":277.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.555","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":251.83,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.564","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":268.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.573","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":249.02,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.408014,"monotonic":2350094.874519177,"hcas":{"mlx5_0":{"port_xmit_data":294442540831329,"port_rcv_data":294785983900544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826207980916,"port_rcv_data":1236934765368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542973492319,"port_rcv_data":294749716588139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898971422646,"port_rcv_data":1193296230389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.511","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.522","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.533","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.544","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.554","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":104.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.563","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":87.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.572","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":86.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.581","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":111.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.4088328,"monotonic":2350095.875338105,"hcas":{"mlx5_0":{"port_xmit_data":294442540831329,"port_rcv_data":294785983900544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826207980916,"port_rcv_data":1236934765368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542973492319,"port_rcv_data":294749716588139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898971422646,"port_rcv_data":1193296230389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.504","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.515","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.526","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.536","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.546","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.555","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.564","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.573","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.4096634,"monotonic":2350096.876168629,"hcas":{"mlx5_0":{"port_xmit_data":294442540831329,"port_rcv_data":294785983900544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826207980916,"port_rcv_data":1236934765368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542973492319,"port_rcv_data":294749716588139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898971422646,"port_rcv_data":1193296230389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.489","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.78,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.511","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:40.522","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:40.532","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.541","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.551","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.560","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.4105163,"monotonic":2350097.877021023,"hcas":{"mlx5_0":{"port_xmit_data":294442541000028,"port_rcv_data":294786043566914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826208167997,"port_rcv_data":1236994450352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542973492319,"port_rcv_data":294749716588139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898971422646,"port_rcv_data":1193296230389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.509","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":176.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.520","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":232.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.531","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":216.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.542","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":209.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.552","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.562","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.571","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.580","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.4113429,"monotonic":2350098.877848144,"hcas":{"mlx5_0":{"port_xmit_data":294442656045826,"port_rcv_data":294786101443278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826323215858,"port_rcv_data":1237052311392,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294542973492319,"port_rcv_data":294749716588139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1898971422646,"port_rcv_data":1193296230389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.513","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":283.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.528","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":279.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.539","gpu_util_pct":91.0,"memory_used_mib":65997.0,"power_w":258.25,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.551","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":278.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.561","gpu_util_pct":30.0,"memory_used_mib":70475.0,"power_w":201.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.570","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":195.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.580","gpu_util_pct":37.0,"memory_used_mib":70475.0,"power_w":203.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.590","gpu_util_pct":96.0,"memory_used_mib":70475.0,"power_w":201.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.4115865,"monotonic":2350099.878090766,"hcas":{"mlx5_0":{"port_xmit_data":294442776660303,"port_rcv_data":294786101573344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826443829778,"port_rcv_data":1237052439890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543385930821,"port_rcv_data":294749717317803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899383840550,"port_rcv_data":1193296957844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.514","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":128.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.526","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":103.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.536","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":83.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.547","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":111.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.556","gpu_util_pct":46.0,"memory_used_mib":70475.0,"power_w":268.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.566","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":293.41,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.575","gpu_util_pct":55.0,"memory_used_mib":70475.0,"power_w":290.16,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.584","gpu_util_pct":88.0,"memory_used_mib":70475.0,"power_w":263.06,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.4124076,"monotonic":2350100.878912591,"hcas":{"mlx5_0":{"port_xmit_data":294442776660303,"port_rcv_data":294786101573344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826443829778,"port_rcv_data":1237052439890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543385931359,"port_rcv_data":294749717318393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899383840550,"port_rcv_data":1193296957844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":103.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.523","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.534","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.544","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":85.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.554","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":108.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.563","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.572","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":113.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.581","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":87.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.413259,"monotonic":2350101.879764035,"hcas":{"mlx5_0":{"port_xmit_data":294442776660303,"port_rcv_data":294786101573344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826443829778,"port_rcv_data":1237052439890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543385931359,"port_rcv_data":294749717318393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899383840550,"port_rcv_data":1193296957844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.516","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.528","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.45,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.538","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.549","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.559","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.569","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.578","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.587","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.4140623,"monotonic":2350102.880567272,"hcas":{"mlx5_0":{"port_xmit_data":294442776660303,"port_rcv_data":294786101573344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826443829778,"port_rcv_data":1237052439890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543385931359,"port_rcv_data":294749717318393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899383840550,"port_rcv_data":1193296957844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.540","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.551","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.71,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:46.562","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.573","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.583","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.593","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.602","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.612","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.4148645,"monotonic":2350103.881369663,"hcas":{"mlx5_0":{"port_xmit_data":294442776798474,"port_rcv_data":294786161209047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826443971252,"port_rcv_data":1237112078647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543385931359,"port_rcv_data":294749717318393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899383840550,"port_rcv_data":1193296957844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.550","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":136.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.561","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":230.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.573","gpu_util_pct":99.0,"memory_used_mib":65997.0,"power_w":216.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.586","gpu_util_pct":87.0,"memory_used_mib":65879.0,"power_w":217.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.597","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.607","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.617","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.626","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.4156618,"monotonic":2350104.882166459,"hcas":{"mlx5_0":{"port_xmit_data":294442891824444,"port_rcv_data":294786220733706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826559021000,"port_rcv_data":1237171596734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543385931359,"port_rcv_data":294749717318393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899383840550,"port_rcv_data":1193296957844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.536","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":256.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.548","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":265.17,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.560","gpu_util_pct":100.0,"memory_used_mib":65997.0,"power_w":255.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.571","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":273.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.583","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":210.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.593","gpu_util_pct":83.0,"memory_used_mib":70475.0,"power_w":199.31,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.602","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":195.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.612","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":214.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.4164486,"monotonic":2350105.882953493,"hcas":{"mlx5_0":{"port_xmit_data":294443009741906,"port_rcv_data":294786220860228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826676937872,"port_rcv_data":1237171723694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543590318107,"port_rcv_data":294749717689726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899588237535,"port_rcv_data":1193297339291,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.498","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":149.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.510","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":166.75,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.521","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":109.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.532","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":151.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.542","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":279.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.552","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":271.75,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.561","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":274.01,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.570","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":292.75,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.4172864,"monotonic":2350106.883790887,"hcas":{"mlx5_0":{"port_xmit_data":294443009746586,"port_rcv_data":294786220861468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826676942552,"port_rcv_data":1237171724934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543798155147,"port_rcv_data":294749718023422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899796073165,"port_rcv_data":1193297697276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.534","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.546","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":116.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.557","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":107.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.568","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":101.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.579","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":87.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.589","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.598","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":90.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.608","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":104.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.4180858,"monotonic":2350107.884591153,"hcas":{"mlx5_0":{"port_xmit_data":294443009746586,"port_rcv_data":294786220861468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826676942552,"port_rcv_data":1237171724934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543798155147,"port_rcv_data":294749718023422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899796073165,"port_rcv_data":1193297697276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.521","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.23,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.532","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.49,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.544","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:51.554","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.564","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":87.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.574","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.583","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.593","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.4189026,"monotonic":2350108.885407901,"hcas":{"mlx5_0":{"port_xmit_data":294443009746586,"port_rcv_data":294786220861468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826676942552,"port_rcv_data":1237171724934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543798155147,"port_rcv_data":294749718023422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899796073165,"port_rcv_data":1193297697276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:52.498","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.23,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:52.510","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:52.522","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.532","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.542","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.552","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.561","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.571","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.4197626,"monotonic":2350109.886267854,"hcas":{"mlx5_0":{"port_xmit_data":294443131258882,"port_rcv_data":294786220992655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826798453928,"port_rcv_data":1237171855863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543920495248,"port_rcv_data":294749718157836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899918412613,"port_rcv_data":1193297829228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.561","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.573","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.62,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.584","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":80.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.594","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.604","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.613","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.622","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.631","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.4205396,"monotonic":2350110.887044666,"hcas":{"mlx5_0":{"port_xmit_data":294443131258882,"port_rcv_data":294786220992655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826798453928,"port_rcv_data":1237171855863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543920495248,"port_rcv_data":294749718157836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899918412613,"port_rcv_data":1193297829228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.537","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.548","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.11,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.560","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.571","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.581","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.591","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.600","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.609","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.4213374,"monotonic":2350111.88784274,"hcas":{"mlx5_0":{"port_xmit_data":294443131258882,"port_rcv_data":294786220992655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826798453928,"port_rcv_data":1237171855863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543920495248,"port_rcv_data":294749718157836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899918412613,"port_rcv_data":1193297829228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.500","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.512","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.51,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.524","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.534","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.544","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.554","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.563","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.573","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.422189,"monotonic":2350112.888694257,"hcas":{"mlx5_0":{"port_xmit_data":294443131428495,"port_rcv_data":294786280660435,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826798641947,"port_rcv_data":1237231541847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543920495248,"port_rcv_data":294749718157836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899918412613,"port_rcv_data":1193297829228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.506","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":178.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.519","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":196.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:56.530","gpu_util_pct":90.0,"memory_used_mib":65997.0,"power_w":195.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.540","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":201.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.550","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.559","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.569","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.577","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.4230325,"monotonic":2350113.889537298,"hcas":{"mlx5_0":{"port_xmit_data":294443246464491,"port_rcv_data":294786339264675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826913676317,"port_rcv_data":1237290129448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294543920495248,"port_rcv_data":294749718157836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1899918412613,"port_rcv_data":1193297829228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.527","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":284.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.540","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":294.72,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.551","gpu_util_pct":96.0,"memory_used_mib":65997.0,"power_w":262.77,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.561","gpu_util_pct":100.0,"memory_used_mib":65879.0,"power_w":279.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.571","gpu_util_pct":40.0,"memory_used_mib":70475.0,"power_w":218.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.582","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":206.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.593","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":213.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.604","gpu_util_pct":100.0,"memory_used_mib":70475.0,"power_w":230.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.4238336,"monotonic":2350114.89033887,"hcas":{"mlx5_0":{"port_xmit_data":294443246464491,"port_rcv_data":294786339264675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826913676317,"port_rcv_data":1237290129448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544212594436,"port_rcv_data":294749718748084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900210492034,"port_rcv_data":1193298421790,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.526","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":124.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.538","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":113.72,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.549","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":92.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.560","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":88.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.570","gpu_util_pct":69.0,"memory_used_mib":70475.0,"power_w":275.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.579","gpu_util_pct":4.0,"memory_used_mib":70475.0,"power_w":271.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.589","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":296.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.598","gpu_util_pct":97.0,"memory_used_mib":70475.0,"power_w":271.06,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.424644,"monotonic":2350115.891149579,"hcas":{"mlx5_0":{"port_xmit_data":294443246464491,"port_rcv_data":294786339264675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826913676317,"port_rcv_data":1237290129448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544212594436,"port_rcv_data":294749718748084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900210492034,"port_rcv_data":1193298421790,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.521","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.533","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.544","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.555","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":88.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.565","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":104.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.575","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":84.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.584","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":109.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.594","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":105.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.4254587,"monotonic":2350116.891963996,"hcas":{"mlx5_0":{"port_xmit_data":294443246464491,"port_rcv_data":294786339264675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826913676317,"port_rcv_data":1237290129448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544212594436,"port_rcv_data":294749718748084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900210492034,"port_rcv_data":1193298421790,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:00.504","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:00.516","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.01,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.528","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.539","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.549","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.559","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.569","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.579","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.4262946,"monotonic":2350117.892799735,"hcas":{"mlx5_0":{"port_xmit_data":294443246464491,"port_rcv_data":294786339264675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1826913676317,"port_rcv_data":1237290129448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544212594436,"port_rcv_data":294749718748084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900210492034,"port_rcv_data":1193298421790,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.506","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.518","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":83.32,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.529","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.540","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.550","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.560","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.570","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.579","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.4271462,"monotonic":2350118.893651054,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.550","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.561","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.34,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.572","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.583","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.592","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.602","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.611","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.620","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.4279442,"monotonic":2350119.8944493,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.573","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.582","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.590","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.599","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.607","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.615","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.623","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.632","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.4287298,"monotonic":2350120.895235372,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.576","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.588","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.599","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.610","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.621","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.631","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.641","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.650","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.4294934,"monotonic":2350121.895998982,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.569","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.581","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.77,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:05.592","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.603","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.613","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.623","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.633","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.642","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.4302666,"monotonic":2350122.896772,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.573","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.584","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.595","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.606","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.616","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.625","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.634","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.643","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.431039,"monotonic":2350123.897544427,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.567","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.579","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.590","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.601","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.610","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.620","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.630","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.639","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.4318163,"monotonic":2350124.898322053,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:08.548","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:08.560","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.571","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.582","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.592","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.602","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.612","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":83.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.621","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.4326136,"monotonic":2350125.899119216,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.552","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.563","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.47,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.575","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.585","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.595","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.605","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.614","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.624","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.4334083,"monotonic":2350126.89991374,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.560","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.572","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":82.23,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.583","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.594","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.604","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.614","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.624","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.633","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.4341967,"monotonic":2350127.900701861,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.572","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:11.583","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.595","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.606","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.616","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.626","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.635","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.645","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.4349718,"monotonic":2350128.901477294,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:12.576","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.588","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.599","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.610","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.620","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.630","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.640","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.649","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.4357445,"monotonic":2350129.90224994,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.552","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.564","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.575","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.586","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.596","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.605","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.615","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.624","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.4365408,"monotonic":2350130.903046357,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.576","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.588","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.599","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.610","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.620","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.630","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.639","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.649","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.4373133,"monotonic":2350131.90381884,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.584","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.596","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.607","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.618","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.628","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.638","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.648","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.657","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.4380782,"monotonic":2350132.904583683,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:16.560","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:16.572","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:16.583","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.593","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.603","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.613","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.623","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.632","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.4388714,"monotonic":2350133.905376447,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.583","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.595","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.606","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.617","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.627","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.637","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.646","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.655","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.4396691,"monotonic":2350134.906173618,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:18.589","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:18.601","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.612","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.623","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.633","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.643","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.652","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.662","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.4404578,"monotonic":2350135.906962723,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.581","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.98,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.603","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.614","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.624","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.633","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.642","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.651","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.4412427,"monotonic":2350136.907747707,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.589","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.601","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.612","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.622","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.632","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.641","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.651","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.660","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.16,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.442038,"monotonic":2350137.908542837,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.582","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.593","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.604","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.615","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.624","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.634","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.643","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.652","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.4428236,"monotonic":2350138.909328723,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:22.582","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:22.593","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.605","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.615","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.625","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.635","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.645","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.654","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.4436042,"monotonic":2350139.910109662,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.589","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.601","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.612","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.623","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.633","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.643","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.652","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.661","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.4443789,"monotonic":2350140.910884311,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.593","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":79.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.604","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.615","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.626","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.636","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.646","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.655","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.664","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.4451656,"monotonic":2350141.911670477,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.603","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.25,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.614","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.625","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.635","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.645","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.654","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.663","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.4455478,"monotonic":2350142.912051875,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.585","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.596","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.608","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.618","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.628","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.638","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.647","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.656","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.4463289,"monotonic":2350143.912833717,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:27.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.604","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.615","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.626","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.636","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.645","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.655","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.664","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.4471,"monotonic":2350144.913605405,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.589","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.600","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.611","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.621","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.631","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.641","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.650","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.659","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.447878,"monotonic":2350145.914383055,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.604","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.615","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.626","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.635","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.645","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.655","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.664","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.448649,"monotonic":2350146.915154408,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:30.596","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:30.607","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.619","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.629","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.639","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.649","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.658","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.667","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.4494176,"monotonic":2350147.915922972,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.603","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.25,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.615","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.625","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.636","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.647","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.658","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.669","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.4501839,"monotonic":2350148.916689172,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.597","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.609","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.49,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.620","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.631","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.641","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.651","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.660","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.670","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.4509525,"monotonic":2350149.917457697,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.604","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.616","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.626","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.636","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.646","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.655","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.665","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.4517248,"monotonic":2350150.918230387,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.596","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.607","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.0,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.619","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.629","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.639","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.649","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.658","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.667","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.4524944,"monotonic":2350151.918999698,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.603","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.76,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.615","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.626","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.636","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.646","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.656","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.665","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.4532669,"monotonic":2350152.919772319,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.600","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.612","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.623","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.634","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.644","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.654","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.663","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.673","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.454031,"monotonic":2350153.920536527,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.600","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.611","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.5,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.622","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.633","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.643","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.652","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.662","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.671","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.4548,"monotonic":2350154.921305434,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.593","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.604","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.25,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.616","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.627","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.637","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.647","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":79.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.656","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.666","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.4555748,"monotonic":2350155.922080216,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.596","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.607","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.619","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.629","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.639","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.649","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.659","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:39.668","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.4563465,"monotonic":2350156.922851969,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.580","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.592","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.603","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.613","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.624","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.633","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:40.643","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":81.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.652","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.4571385,"monotonic":2350157.923643671,"hcas":{"mlx5_0":{"port_xmit_data":294443366181745,"port_rcv_data":294786339392306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1827033392781,"port_rcv_data":1237290255866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":294544333139658,"port_rcv_data":294749718879152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1900331036570,"port_rcv_data":1193298551045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.600","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":78.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.611","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":81.01,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.622","gpu_util_pct":0.0,"memory_used_mib":65997.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.633","gpu_util_pct":0.0,"memory_used_mib":65879.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.643","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.653","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":78.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.662","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":82.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.671","gpu_util_pct":0.0,"memory_used_mib":70475.0,"power_w":80.57,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_3.jsonl

@@ -0,0 +1,460 @@
+{"wall_time":1788258843.1803372,"monotonic":2349699.545973118,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:03.307","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:03.323","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:03.334","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:03.344","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:03.353","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:03.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:03.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:03.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258844.1811295,"monotonic":2349700.546765139,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:04.332","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:04.346","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:04.356","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:04.367","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:04.376","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:04.386","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:04.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:04.403","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258845.1819055,"monotonic":2349701.547539914,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.307","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.318","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.329","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.340","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.349","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.359","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.368","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.377","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.1827052,"monotonic":2349702.548340544,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.330","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.375","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.382","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.183497,"monotonic":2349703.549132236,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.299","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.310","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.320","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.331","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.340","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.349","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.358","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.366","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.184305,"monotonic":2349704.549940631,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.317","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.338","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.358","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.376","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.385","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.1850996,"monotonic":2349705.550735231,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.333","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.344","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.354","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.374","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.383","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.392","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.400","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.1858776,"monotonic":2349706.551513111,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.329","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.340","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.351","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.361","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.398","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.1864045,"monotonic":2349707.552039858,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.329","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.340","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.351","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.361","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.379","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.388","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.1871898,"monotonic":2349708.55282533,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.330","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.363","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.373","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.382","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.391","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.400","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.187971,"monotonic":2349709.553606957,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.329","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.340","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.350","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.361","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.188759,"monotonic":2349710.554394923,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.339","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.350","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.370","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.379","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.396","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.1893392,"monotonic":2349711.554974238,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.312","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.324","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.336","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.346","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.358","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.368","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.1901336,"monotonic":2349712.555769029,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.333","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.344","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.355","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.365","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.375","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.384","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.392","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.401","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.1903892,"monotonic":2349713.556024911,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.337","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.348","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.358","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.368","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.396","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.1911647,"monotonic":2349714.556800452,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.313","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.324","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.334","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.344","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.354","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.363","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.372","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.19197,"monotonic":2349715.557605747,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.338","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.370","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.390","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.399","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.408","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.1927445,"monotonic":2349716.558380263,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.329","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.340","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.351","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.362","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.381","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.390","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.398","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.1935291,"monotonic":2349717.559164977,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.374","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.384","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.394","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.403","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.412","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.1943,"monotonic":2349718.559935763,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.338","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.359","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.370","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.379","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.388","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.406","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.1943839,"monotonic":2349719.560019721,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.363","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.92,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.373","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.383","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.392","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.401","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.410","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.195159,"monotonic":2349720.560794768,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.363","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.68,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.373","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.383","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.392","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.401","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.410","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.1959357,"monotonic":2349721.561571467,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.342","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.354","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.365","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.375","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.385","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.413","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.19673,"monotonic":2349722.562365173,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.346","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.357","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.368","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.378","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.406","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.414","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.1975067,"monotonic":2349723.563142406,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.321","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.332","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.342","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":80.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.379","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.388","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.198313,"monotonic":2349724.563948372,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.333","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.344","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.355","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.366","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.376","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.385","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.394","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.403","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.1984015,"monotonic":2349725.564037074,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.348","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.371","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.381","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.390","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.400","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.409","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.418","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.1991854,"monotonic":2349726.564820381,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.318","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.329","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.339","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.350","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.360","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.369","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.1999943,"monotonic":2349727.565630217,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.363","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.373","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.382","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.392","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.401","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.410","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.2007804,"monotonic":2349728.566415977,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.322","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.332","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.343","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.388","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.2015886,"monotonic":2349729.567224587,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.370","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.380","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.398","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.407","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.415","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.2023795,"monotonic":2349730.568015187,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.348","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.359","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.370","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.380","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.390","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.399","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.408","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.417","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.2031605,"monotonic":2349731.568796295,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.371","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.381","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.390","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.400","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.409","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.418","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.2039416,"monotonic":2349732.569577473,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.343","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.354","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.365","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.69,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.375","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.385","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.394","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.412","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.2047322,"monotonic":2349733.570367823,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.374","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.69,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.384","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.394","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.403","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.412","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:37.420","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.2055135,"monotonic":2349734.571149199,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.374","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.384","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.394","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.403","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.412","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.421","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.2062986,"monotonic":2349735.571933972,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.354","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.365","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.375","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.386","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.413","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.422","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.2070768,"monotonic":2349736.572712534,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.374","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.385","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":80.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.413","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.422","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.2078624,"monotonic":2349737.573498006,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.375","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.385","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":80.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.413","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:41.422","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.2086413,"monotonic":2349738.574277186,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.350","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.361","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.372","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.382","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.392","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.402","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.411","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.420","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.2094185,"monotonic":2349739.575054548,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.374","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.385","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.413","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.422","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.2102,"monotonic":2349740.575835846,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.340","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.351","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.370","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.379","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.388","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.2110014,"monotonic":2349741.576637405,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.375","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.386","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.396","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.406","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.416","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.425","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.434","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.2117739,"monotonic":2349742.577409619,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.364","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.375","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.385","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.414","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.422","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.212569,"monotonic":2349743.578204839,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.292","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.303","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.314","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.325","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.335","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.344","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.353","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:47.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.2134109,"monotonic":2349744.57904652,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.291","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.302","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.313","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.323","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.333","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.342","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.351","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.360","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.2142563,"monotonic":2349745.579892102,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.316","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.327","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.337","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.348","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.358","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.377","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.386","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.2150753,"monotonic":2349746.580711003,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.294","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.305","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.315","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.325","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.335","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.344","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.354","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.2159233,"monotonic":2349747.581558868,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.316","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.327","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.338","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.92,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.348","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.357","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.376","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:51.385","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.2167475,"monotonic":2349748.582383208,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.316","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.338","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.68,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.348","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.358","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.376","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.384","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.21757,"monotonic":2349749.583205936,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.316","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.327","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.338","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.348","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.358","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.376","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.385","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.218407,"monotonic":2349750.584042401,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.342","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.363","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.373","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.382","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.391","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.400","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.409","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.21921,"monotonic":2349751.584845682,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.299","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.309","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.320","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.330","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.340","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.349","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.358","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.2200544,"monotonic":2349752.585690101,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.321","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.332","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.342","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.353","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.372","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.381","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.2208776,"monotonic":2349753.586513528,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.299","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.310","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.321","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.331","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.340","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.350","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.359","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.367","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.2217247,"monotonic":2349754.587360223,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.320","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.331","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.372","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.381","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.390","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.2225509,"monotonic":2349755.588186668,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.320","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.331","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.342","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.352","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.2233775,"monotonic":2349756.589013093,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.346","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.357","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.367","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.377","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.406","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.415","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.2241795,"monotonic":2349757.589815064,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.318","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.337","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.346","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.355","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.363","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.372","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.225016,"monotonic":2349758.590652018,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.339","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.359","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.369","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.396","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.2258384,"monotonic":2349759.591474319,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.305","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.316","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.326","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.337","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.347","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.356","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.365","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.374","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.2266824,"monotonic":2349760.592318261,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.339","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.360","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.370","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.379","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.388","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.227503,"monotonic":2349761.593139161,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.306","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.317","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.338","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.348","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.357","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.366","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.375","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.228348,"monotonic":2349762.593983809,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.328","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.339","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.349","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.359","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.369","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.396","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.2291734,"monotonic":2349763.594809368,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.309","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.320","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.331","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.350","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.359","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.368","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.377","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.230017,"monotonic":2349764.595652939,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.309","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.320","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.331","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.341","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.351","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.360","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.370","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.2308636,"monotonic":2349765.596499675,"hcas":{"mlx5_0":{"port_xmit_data":350023859997714,"port_rcv_data":350254041426770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952022052,"port_rcv_data":1321943444290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.310","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.321","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":77.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.331","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.342","gpu_util_pct":0.0,"memory_used_mib":61791.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.352","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.362","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.2317197,"monotonic":2349766.597355272,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.329","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":93.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.341","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":92.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.352","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":97.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.363","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":95.09,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.373","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":76.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.382","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.392","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.401","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.2325447,"monotonic":2349767.598180529,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.334","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":104.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.345","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":101.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.356","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":107.01,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.367","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":106.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.377","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.386","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:11.396","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:11.405","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.233372,"monotonic":2349768.599007982,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.335","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.346","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":77.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.357","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":81.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.368","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.406","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.2342007,"monotonic":2349769.599836438,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.334","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.346","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.357","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.367","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.377","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.386","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.235025,"monotonic":2349770.600659962,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.334","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.346","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":77.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.357","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":81.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.367","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.377","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.386","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.395","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.404","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.2358494,"monotonic":2349771.601485075,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.317","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.332","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":77.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.344","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.356","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.368","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.378","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:15.398","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.2366774,"monotonic":2349772.602313191,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.334","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.346","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.357","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.367","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.377","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.387","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.396","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.405","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.237503,"monotonic":2349773.603138911,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.338","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.349","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":77.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.360","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.370","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.399","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.408","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.2383258,"monotonic":2349774.603961657,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.318","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.330","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":77.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.341","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":81.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.351","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.361","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.371","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.380","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.389","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":78.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.238407,"monotonic":2349775.6040426,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.354","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.365","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.376","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.387","gpu_util_pct":0.0,"memory_used_mib":69931.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.397","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:19.407","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:19.416","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:19.425","gpu_util_pct":0.0,"memory_used_mib":61875.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.2392135,"monotonic":2349776.604849217,"hcas":{"mlx5_0":{"port_xmit_data":350023860199676,"port_rcv_data":350254130875270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1848952212134,"port_rcv_data":1322032881545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985644895589,"port_rcv_data":350348826121571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944760922586,"port_rcv_data":1255383792847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.343","gpu_util_pct":100.0,"memory_used_mib":69931.0,"power_w":163.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.356","gpu_util_pct":100.0,"memory_used_mib":69931.0,"power_w":187.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.367","gpu_util_pct":100.0,"memory_used_mib":69931.0,"power_w":211.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.377","gpu_util_pct":100.0,"memory_used_mib":69931.0,"power_w":198.37,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.387","gpu_util_pct":0.0,"memory_used_mib":64531.0,"power_w":77.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.409","gpu_util_pct":74.0,"memory_used_mib":64675.0,"power_w":80.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.428","gpu_util_pct":0.0,"memory_used_mib":65035.0,"power_w":79.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.445","gpu_util_pct":85.0,"memory_used_mib":65083.0,"power_w":79.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.2399843,"monotonic":2349777.605619789,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985751375577,"port_rcv_data":350348826354535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944867401566,"port_rcv_data":1255384025919,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.357","gpu_util_pct":97.0,"memory_used_mib":70567.0,"power_w":299.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.368","gpu_util_pct":100.0,"memory_used_mib":70567.0,"power_w":286.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.379","gpu_util_pct":100.0,"memory_used_mib":70567.0,"power_w":291.28,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.390","gpu_util_pct":100.0,"memory_used_mib":70567.0,"power_w":293.6,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.399","gpu_util_pct":43.0,"memory_used_mib":71093.0,"power_w":241.05,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.409","gpu_util_pct":99.0,"memory_used_mib":71093.0,"power_w":246.41,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:21.419","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":168.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:21.429","gpu_util_pct":99.0,"memory_used_mib":71093.0,"power_w":267.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.2407868,"monotonic":2349778.606422293,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.358","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.369","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.380","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":83.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.391","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":80.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.400","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":274.96,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:22.410","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":283.17,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.419","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":286.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.429","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":264.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.2415938,"monotonic":2349779.607229583,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.321","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.332","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.343","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":82.82,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.354","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.363","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":102.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.373","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":106.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.382","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":111.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.391","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":105.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.2424378,"monotonic":2349780.608073835,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.322","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.333","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.344","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.354","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.364","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.374","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:24.383","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:24.392","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.2432847,"monotonic":2349781.608920728,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.344","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.356","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.367","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.377","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.387","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.397","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.407","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.416","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.2441065,"monotonic":2349782.609742472,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.340","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.351","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.362","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.372","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.382","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.391","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.400","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.409","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.2449358,"monotonic":2349783.610571351,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.344","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.355","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.366","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.377","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.386","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.396","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.405","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:27.414","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.24576,"monotonic":2349784.611395798,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.361","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.372","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.383","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.69,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.393","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.403","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.412","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.421","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.430","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.2465687,"monotonic":2349785.612204537,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.331","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.342","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.353","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.364","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.374","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.384","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:29.393","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:29.402","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.24741,"monotonic":2349786.613046094,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.326","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.338","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.348","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.359","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.369","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.379","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.388","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.397","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.2482631,"monotonic":2349787.613898626,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985879028217,"port_rcv_data":350348826492490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1944995054239,"port_rcv_data":1255384163720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.349","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.360","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.371","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.381","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.391","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.400","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":78.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.410","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.419","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.2490964,"monotonic":2349788.614732006,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985991808973,"port_rcv_data":350348826805397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945107824337,"port_rcv_data":1255384489754,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.374","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.385","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.396","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.407","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.416","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.426","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":81.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.436","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.445","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":82.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.249904,"monotonic":2349789.615539406,"hcas":{"mlx5_0":{"port_xmit_data":350023975323974,"port_rcv_data":350254215588715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849067322320,"port_rcv_data":1322117554846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349985991808973,"port_rcv_data":350348826805397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945107824337,"port_rcv_data":1255384489754,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.369","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.380","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.391","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.402","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.412","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.422","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.431","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":81.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.441","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":81.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.250408,"monotonic":2349790.61604364,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.391","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.403","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.413","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.83,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.424","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.434","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.444","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:34.454","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:34.463","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.2511916,"monotonic":2349791.616827491,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.373","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.385","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.396","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":82.19,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.406","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.416","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.426","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.435","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.444","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.28,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.2519946,"monotonic":2349792.617630452,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.373","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.384","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.395","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.406","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.416","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.426","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:36.435","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:36.444","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.19,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.2527974,"monotonic":2349793.618433159,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.332","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.344","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.354","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.365","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.375","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.385","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.394","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.404","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.2536473,"monotonic":2349794.619283133,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.396","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.408","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.419","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.87,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.429","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.439","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.449","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.458","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.8,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.467","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.58,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.254433,"monotonic":2349795.620068831,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.401","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.413","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.424","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.97,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.434","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.443","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.453","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.462","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":80.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.471","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.2552161,"monotonic":2349796.620851787,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.357","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.368","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.379","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":81.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.389","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.399","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.409","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:40.418","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.427","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.2560403,"monotonic":2349797.621676189,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.357","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.368","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.379","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.390","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.400","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:41.409","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.418","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.427","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.2568662,"monotonic":2349798.622502068,"hcas":{"mlx5_0":{"port_xmit_data":350024091381828,"port_rcv_data":350254215712796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183379384,"port_rcv_data":1322117679782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.360","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":79.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.372","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.383","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":80.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.393","gpu_util_pct":0.0,"memory_used_mib":70567.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:42.402","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.412","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.421","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.430","gpu_util_pct":0.0,"memory_used_mib":71093.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.25769,"monotonic":2349799.623325664,"hcas":{"mlx5_0":{"port_xmit_data":350024091635279,"port_rcv_data":350254305213606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849183630330,"port_rcv_data":1322207178072,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986107866755,"port_rcv_data":350348826929044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945223881401,"port_rcv_data":1255384613398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.374","gpu_util_pct":9.0,"memory_used_mib":70679.0,"power_w":177.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.386","gpu_util_pct":0.0,"memory_used_mib":70679.0,"power_w":160.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.397","gpu_util_pct":43.0,"memory_used_mib":70679.0,"power_w":162.52,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.406","gpu_util_pct":100.0,"memory_used_mib":70679.0,"power_w":168.56,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.414","gpu_util_pct":0.0,"memory_used_mib":71205.0,"power_w":77.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.423","gpu_util_pct":0.0,"memory_used_mib":71205.0,"power_w":79.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.430","gpu_util_pct":0.0,"memory_used_mib":71205.0,"power_w":80.38,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.439","gpu_util_pct":77.0,"memory_used_mib":71205.0,"power_w":79.57,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.258497,"monotonic":2349800.624132881,"hcas":{"mlx5_0":{"port_xmit_data":350024206774639,"port_rcv_data":350254411463633,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849298783433,"port_rcv_data":1322321174148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986342082928,"port_rcv_data":350348827384245,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945458102450,"port_rcv_data":1255385074030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.360","gpu_util_pct":94.0,"memory_used_mib":71127.0,"power_w":288.96,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.370","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":285.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.380","gpu_util_pct":95.0,"memory_used_mib":71127.0,"power_w":322.42,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.390","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":274.56,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.401","gpu_util_pct":51.0,"memory_used_mib":71205.0,"power_w":186.22,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.410","gpu_util_pct":99.0,"memory_used_mib":71205.0,"power_w":178.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.419","gpu_util_pct":3.0,"memory_used_mib":71205.0,"power_w":203.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.429","gpu_util_pct":100.0,"memory_used_mib":71205.0,"power_w":282.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.2593088,"monotonic":2349801.624944608,"hcas":{"mlx5_0":{"port_xmit_data":350024322031391,"port_rcv_data":350254568921394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849414030442,"port_rcv_data":1322470862356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986454810253,"port_rcv_data":350348827648960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945570787176,"port_rcv_data":1255385320567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.369","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":283.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.382","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":261.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.394","gpu_util_pct":99.0,"memory_used_mib":71127.0,"power_w":265.4,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.406","gpu_util_pct":51.0,"memory_used_mib":71127.0,"power_w":245.08,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:45.416","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":318.63,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:45.427","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":339.35,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:45.437","gpu_util_pct":98.0,"memory_used_mib":71765.0,"power_w":325.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.447","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":334.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.2601013,"monotonic":2349802.625737274,"hcas":{"mlx5_0":{"port_xmit_data":350024437037870,"port_rcv_data":350254613019214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849529031494,"port_rcv_data":1322514936847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986688922828,"port_rcv_data":350348827999892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1945804900206,"port_rcv_data":1255385673815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.363","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":335.79,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.375","gpu_util_pct":98.0,"memory_used_mib":71127.0,"power_w":308.27,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.387","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":340.3,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.399","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":307.67,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.410","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":269.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.421","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":229.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.432","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":264.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.441","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":221.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.2609062,"monotonic":2349803.626541964,"hcas":{"mlx5_0":{"port_xmit_data":350024437269513,"port_rcv_data":350254702497473,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849529264197,"port_rcv_data":1322604416475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986923113513,"port_rcv_data":350348828431286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1946039085850,"port_rcv_data":1255386100091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.384","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":188.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.396","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":239.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.408","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":244.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.420","gpu_util_pct":97.0,"memory_used_mib":71127.0,"power_w":205.77,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.430","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":293.41,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.441","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":308.95,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.451","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":298.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.461","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":306.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.2616947,"monotonic":2349804.627330554,"hcas":{"mlx5_0":{"port_xmit_data":350024552338816,"port_rcv_data":350254792046574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849644360982,"port_rcv_data":1322693992733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349986981758519,"port_rcv_data":350348828601645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1946097695793,"port_rcv_data":1255386256207,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.365","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":249.5,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.377","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":251.39,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.388","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":254.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.399","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":292.2,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:48.409","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":243.57,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.420","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":274.14,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.431","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":248.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.442","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":247.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.2624984,"monotonic":2349805.628134215,"hcas":{"mlx5_0":{"port_xmit_data":350024667246639,"port_rcv_data":350254800455570,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849759269526,"port_rcv_data":1322702383655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349987215969935,"port_rcv_data":350348829052985,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1946331906525,"port_rcv_data":1255386707725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.344","gpu_util_pct":98.0,"memory_used_mib":71127.0,"power_w":246.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.356","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":263.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.367","gpu_util_pct":98.0,"memory_used_mib":71127.0,"power_w":261.27,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.378","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":239.5,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.388","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":198.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.400","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":192.22,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.411","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":219.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.421","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":218.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.2633247,"monotonic":2349806.628960368,"hcas":{"mlx5_0":{"port_xmit_data":350024667466916,"port_rcv_data":350254889922550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849759496777,"port_rcv_data":1322791857847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349987450165645,"port_rcv_data":350348829493453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1946566094976,"port_rcv_data":1255387142671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.364","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":177.51,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.377","gpu_util_pct":87.0,"memory_used_mib":71127.0,"power_w":184.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.388","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":173.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.399","gpu_util_pct":97.0,"memory_used_mib":71127.0,"power_w":141.1,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.410","gpu_util_pct":58.0,"memory_used_mib":71765.0,"power_w":250.54,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.420","gpu_util_pct":0.0,"memory_used_mib":71765.0,"power_w":295.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.429","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":310.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.438","gpu_util_pct":0.0,"memory_used_mib":71765.0,"power_w":304.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.2641385,"monotonic":2349807.629774173,"hcas":{"mlx5_0":{"port_xmit_data":350024898574173,"port_rcv_data":350254975490717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849990601640,"port_rcv_data":1322877392598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349987461218175,"port_rcv_data":350348829529203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1946577140054,"port_rcv_data":1255387179301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.383","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":258.36,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.394","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":236.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.405","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":257.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:51.416","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":244.58,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:51.426","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":173.94,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.437","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":192.86,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.447","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":145.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.459","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":195.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.2649221,"monotonic":2349808.630558023,"hcas":{"mlx5_0":{"port_xmit_data":350024898782302,"port_rcv_data":350255064942620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1849990824379,"port_rcv_data":1322966858295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349987811466337,"port_rcv_data":350348830085417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1946927383859,"port_rcv_data":1255387732021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.380","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":220.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.390","gpu_util_pct":99.0,"memory_used_mib":71127.0,"power_w":220.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.400","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":253.2,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.411","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":227.18,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.422","gpu_util_pct":98.0,"memory_used_mib":71765.0,"power_w":168.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.431","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":171.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.440","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":198.84,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.449","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":151.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.2657232,"monotonic":2349809.63135909,"hcas":{"mlx5_0":{"port_xmit_data":350025013886379,"port_rcv_data":350255154522615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850105935093,"port_rcv_data":1323056444421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349987811466875,"port_rcv_data":350348830086007,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1946927383859,"port_rcv_data":1255387732021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.387","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":257.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.400","gpu_util_pct":99.0,"memory_used_mib":71127.0,"power_w":249.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.410","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":258.98,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.420","gpu_util_pct":67.0,"memory_used_mib":71127.0,"power_w":246.46,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.430","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":292.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.441","gpu_util_pct":97.0,"memory_used_mib":71765.0,"power_w":287.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.451","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":267.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.461","gpu_util_pct":97.0,"memory_used_mib":71765.0,"power_w":300.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.2665303,"monotonic":2349810.632165498,"hcas":{"mlx5_0":{"port_xmit_data":350025201910403,"port_rcv_data":350255199734474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850275905091,"port_rcv_data":1323101590637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349988159447459,"port_rcv_data":350348830766201,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1947275316596,"port_rcv_data":1255388394030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.372","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":308.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.385","gpu_util_pct":98.0,"memory_used_mib":71127.0,"power_w":320.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.397","gpu_util_pct":98.0,"memory_used_mib":71127.0,"power_w":361.13,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.407","gpu_util_pct":93.0,"memory_used_mib":71127.0,"power_w":295.7,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.419","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":255.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.429","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":275.44,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.440","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":261.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.450","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":316.92,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.2673383,"monotonic":2349811.632973768,"hcas":{"mlx5_0":{"port_xmit_data":350025202143800,"port_rcv_data":350255289209520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850276156699,"port_rcv_data":1323191085034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349988448635570,"port_rcv_data":350348831285335,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1947582549101,"port_rcv_data":1255388922808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.352","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":237.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.364","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":224.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.376","gpu_util_pct":97.0,"memory_used_mib":71127.0,"power_w":238.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.387","gpu_util_pct":97.0,"memory_used_mib":71127.0,"power_w":245.7,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:55.400","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":321.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.409","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":303.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.419","gpu_util_pct":98.0,"memory_used_mib":71765.0,"power_w":316.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.429","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":314.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.2681699,"monotonic":2349812.633805293,"hcas":{"mlx5_0":{"port_xmit_data":350025317263422,"port_rcv_data":350255377709666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850391255045,"port_rcv_data":1323279554310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349988508639922,"port_rcv_data":350348831435485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1947642512518,"port_rcv_data":1255389079518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.371","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":238.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.385","gpu_util_pct":99.0,"memory_used_mib":71127.0,"power_w":242.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.399","gpu_util_pct":99.0,"memory_used_mib":71127.0,"power_w":213.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.410","gpu_util_pct":99.0,"memory_used_mib":71127.0,"power_w":246.06,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.421","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":239.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.432","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":192.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.442","gpu_util_pct":98.0,"memory_used_mib":71765.0,"power_w":238.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.452","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":214.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.2689693,"monotonic":2349813.634604566,"hcas":{"mlx5_0":{"port_xmit_data":350025335731814,"port_rcv_data":350255467223647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850409727967,"port_rcv_data":1323369071975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349988742806766,"port_rcv_data":350348831837848,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1947876727403,"port_rcv_data":1255389530410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.373","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":262.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.386","gpu_util_pct":97.0,"memory_used_mib":71127.0,"power_w":286.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.398","gpu_util_pct":97.0,"memory_used_mib":71127.0,"power_w":289.17,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.409","gpu_util_pct":98.0,"memory_used_mib":71127.0,"power_w":275.63,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.420","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":227.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.430","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":241.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.440","gpu_util_pct":99.0,"memory_used_mib":71765.0,"power_w":235.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.450","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":263.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.2691681,"monotonic":2349814.634803851,"hcas":{"mlx5_0":{"port_xmit_data":350025450857445,"port_rcv_data":350255554559925,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850524843092,"port_rcv_data":1323456353763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349988878867563,"port_rcv_data":350348832163816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1948012772785,"port_rcv_data":1255389853857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.374","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":269.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.386","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":267.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.398","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":280.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.408","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":272.4,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:58.420","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":320.49,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:58.431","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":315.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:58.441","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":303.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:58.451","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":310.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.269976,"monotonic":2349815.635610993,"hcas":{"mlx5_0":{"port_xmit_data":350025586932997,"port_rcv_data":350255644177302,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850624909377,"port_rcv_data":1323545928517,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349989113087922,"port_rcv_data":350348832619545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1948246998438,"port_rcv_data":1255390314763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.373","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":291.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.385","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":292.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.397","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":292.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.408","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":313.18,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.420","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":245.4,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.430","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":269.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.440","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":234.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.449","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":255.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.2703977,"monotonic":2349816.636032908,"hcas":{"mlx5_0":{"port_xmit_data":350025702012192,"port_rcv_data":350255733729856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850740002300,"port_rcv_data":1323635494360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349989329190568,"port_rcv_data":350348833017414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1948499062530,"port_rcv_data":1255390747428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.375","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":213.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.388","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":212.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.401","gpu_util_pct":100.0,"memory_used_mib":71127.0,"power_w":250.74,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.413","gpu_util_pct":99.0,"memory_used_mib":71127.0,"power_w":191.81,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.424","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":279.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.436","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":289.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.446","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":295.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.457","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":290.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.270931,"monotonic":2349817.636566521,"hcas":{"mlx5_0":{"port_xmit_data":350025816896775,"port_rcv_data":350255734870333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850854885654,"port_rcv_data":1323636595972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349989563396748,"port_rcv_data":350348833463208,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1948733247846,"port_rcv_data":1255391171177,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.371","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":258.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.384","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":238.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.395","gpu_util_pct":0.0,"memory_used_mib":71129.0,"power_w":252.83,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.406","gpu_util_pct":3.0,"memory_used_mib":71129.0,"power_w":244.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.417","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":276.58,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.427","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":255.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:01.437","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":255.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.446","gpu_util_pct":100.0,"memory_used_mib":71765.0,"power_w":238.48,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.2717462,"monotonic":2349818.637381786,"hcas":{"mlx5_0":{"port_xmit_data":350025892018585,"port_rcv_data":350255734954664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1850911352742,"port_rcv_data":1323636661118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349989797579736,"port_rcv_data":350348833885043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1948967437749,"port_rcv_data":1255391597556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.385","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.397","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.409","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.420","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.430","gpu_util_pct":0.0,"memory_used_mib":71767.0,"power_w":185.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.440","gpu_util_pct":0.0,"memory_used_mib":71767.0,"power_w":210.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.449","gpu_util_pct":0.0,"memory_used_mib":71767.0,"power_w":213.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.458","gpu_util_pct":0.0,"memory_used_mib":71767.0,"power_w":219.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.2725542,"monotonic":2349819.638190269,"hcas":{"mlx5_0":{"port_xmit_data":350026032058516,"port_rcv_data":350255735101291,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851014194030,"port_rcv_data":1323636772160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349989855414546,"port_rcv_data":350348833961531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949043888033,"port_rcv_data":1255391690129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.373","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.385","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.397","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.57,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.407","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.418","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.427","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.446","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.2731457,"monotonic":2349820.638781436,"hcas":{"mlx5_0":{"port_xmit_data":350026032063967,"port_rcv_data":350255735103148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851014198646,"port_rcv_data":1323636773400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349989958253024,"port_rcv_data":350348834071444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949183924749,"port_rcv_data":1255391836619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.391","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.404","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.415","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.6,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.427","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.15,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.457","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.466","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.273958,"monotonic":2349821.639593384,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.374","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.386","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.397","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.45,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.407","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.92,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.417","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:05.427","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.436","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.445","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.2743974,"monotonic":2349822.640032917,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.376","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.388","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.399","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.29,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.409","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.81,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.419","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.428","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.446","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.2752254,"monotonic":2349823.640860877,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.398","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.409","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.421","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.98,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.431","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.442","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.451","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.461","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.03,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.470","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.27604,"monotonic":2349824.641675658,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.373","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.385","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.396","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.407","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:08.417","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.427","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.436","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.445","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.2768672,"monotonic":2349825.642502622,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.378","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.390","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.401","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.79,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.412","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.422","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.432","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.441","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.451","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.2775066,"monotonic":2349826.643142049,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.377","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.389","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.400","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.64,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.411","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.421","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.430","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.440","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.449","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.2783298,"monotonic":2349827.643965583,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.381","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.392","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.403","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.414","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.424","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.434","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:11.443","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.452","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.2791512,"monotonic":2349828.644786701,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.376","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.388","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.399","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.409","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:12.419","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:12.429","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.2799778,"monotonic":2349829.645613551,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.398","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.410","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.421","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.32,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.432","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.443","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.452","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.462","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.28,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.471","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.2807815,"monotonic":2349830.646417143,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.377","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.389","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.400","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.36,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.411","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.421","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.430","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.440","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.449","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.281604,"monotonic":2349831.647239671,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.383","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.395","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.406","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.417","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.427","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:15.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:15.446","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.2823896,"monotonic":2349832.648025328,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.384","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.396","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.407","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.418","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.428","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.457","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.2832084,"monotonic":2349833.648844191,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.384","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.396","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.407","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.418","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.428","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.2840257,"monotonic":2349834.649661215,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.385","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.397","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.408","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.12,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.419","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.429","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.19,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.2848446,"monotonic":2349835.650480442,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.381","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.393","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.404","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.5,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.415","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.425","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.435","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.445","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.58,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.454","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.2856736,"monotonic":2349836.651309507,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.423","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.434","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.446","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.07,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.456","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.466","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:20.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.19,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.2864,"monotonic":2349837.652035678,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.392","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.406","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.417","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.29,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.427","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.446","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.49,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.465","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.2872148,"monotonic":2349838.652849839,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.433","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.456","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.01,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:22.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.2879858,"monotonic":2349839.65362171,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.433","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.444","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.2,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.17,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.505","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.2887573,"monotonic":2349840.654392833,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.433","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.444","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:24.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.505","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.65,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.2895296,"monotonic":2349841.655165276,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.438","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.449","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.470","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.480","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.489","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.498","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.507","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.290306,"monotonic":2349842.655941763,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.438","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.449","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.470","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:26.480","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.490","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.509","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.2910779,"monotonic":2349843.656713644,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.438","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.449","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.470","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.480","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.489","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.498","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.507","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.2918553,"monotonic":2349844.657490994,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.393","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.404","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:28.415","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:28.425","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:28.435","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:28.444","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.453","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.462","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.29269,"monotonic":2349845.658325406,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.396","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.407","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.418","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.428","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.455","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.464","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.2935195,"monotonic":2349846.659155397,"hcas":{"mlx5_0":{"port_xmit_data":350026179270361,"port_rcv_data":350255735272350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851124800114,"port_rcv_data":1323636903833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.396","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.408","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.418","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.429","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:30.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:30.448","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.457","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.466","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.2943406,"monotonic":2349847.65997628,"hcas":{"mlx5_0":{"port_xmit_data":350026179530386,"port_rcv_data":350255824779842,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851125071769,"port_rcv_data":1323726422955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.395","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":177.25,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.406","gpu_util_pct":82.0,"memory_used_mib":71131.0,"power_w":165.46,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.417","gpu_util_pct":78.0,"memory_used_mib":71131.0,"power_w":161.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.427","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":164.95,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.446","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.455","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.463","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.2951634,"monotonic":2349848.660799138,"hcas":{"mlx5_0":{"port_xmit_data":350026294634724,"port_rcv_data":350255913215126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851240184191,"port_rcv_data":1323814846740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990068859906,"port_rcv_data":350348834205003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949331131281,"port_rcv_data":1255392005554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.411","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":282.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.424","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":285.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.436","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":298.42,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:32.447","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":277.78,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:32.458","gpu_util_pct":86.0,"memory_used_mib":71769.0,"power_w":158.26,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.467","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":167.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.477","gpu_util_pct":34.0,"memory_used_mib":71769.0,"power_w":164.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.487","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":173.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.2959573,"monotonic":2349849.661592912,"hcas":{"mlx5_0":{"port_xmit_data":350026294634724,"port_rcv_data":350255913215126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851240184191,"port_rcv_data":1323814846740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990303071234,"port_rcv_data":350348834654731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949565342136,"port_rcv_data":1255392455785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.393","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":102.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.404","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.415","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":113.32,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.425","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":132.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.435","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":304.42,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.445","gpu_util_pct":97.0,"memory_used_mib":71769.0,"power_w":290.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.454","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":311.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.463","gpu_util_pct":89.0,"memory_used_mib":71769.0,"power_w":304.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.2967815,"monotonic":2349850.662417286,"hcas":{"mlx5_0":{"port_xmit_data":350026294634724,"port_rcv_data":350255913215126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851240184191,"port_rcv_data":1323814846740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990420788646,"port_rcv_data":350348834919502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949683061403,"port_rcv_data":1255392730324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.376","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":100.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.387","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.398","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.63,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.408","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":109.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:34.417","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":88.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.427","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":86.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.436","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.47,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.445","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.2976234,"monotonic":2349851.663259187,"hcas":{"mlx5_0":{"port_xmit_data":350026294634724,"port_rcv_data":350255913215126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851240184191,"port_rcv_data":1323814846740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990420788646,"port_rcv_data":350348834919502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949683061403,"port_rcv_data":1255392730324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.376","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.387","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.398","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.57,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.408","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.418","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.428","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.446","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.2984383,"monotonic":2349852.664073975,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.396","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.408","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.418","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.97,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.428","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.465","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.2992668,"monotonic":2349853.664902662,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.397","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.408","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.419","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.12,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.429","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.439","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.448","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.458","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.466","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.3000915,"monotonic":2349854.665727256,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.456","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.82,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:38.477","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:38.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.505","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.17,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.514","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.77,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.3008683,"monotonic":2349855.666504012,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.425","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.436","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.446","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.25,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.456","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.466","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.485","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.494","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.3016663,"monotonic":2349856.667302061,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.440","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.451","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.462","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.481","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.490","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.3024511,"monotonic":2349857.668086892,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.444","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:41.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:41.476","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.513","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.3032293,"monotonic":2349858.668865045,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.440","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.452","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.462","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.482","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.492","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.501","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.510","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.3040135,"monotonic":2349859.669649336,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.448","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.459","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.470","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:43.480","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:43.489","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.498","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.507","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.3047931,"monotonic":2349860.670428881,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.440","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.451","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.462","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.481","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.490","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.3055866,"monotonic":2349861.67122248,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.450","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.461","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.482","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.492","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.501","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.510","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.519","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.3063815,"monotonic":2349862.672017414,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.453","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.464","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.474","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.485","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.494","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.513","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.522","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.3071592,"monotonic":2349863.672795263,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.456","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.467","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.477","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.505","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.513","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.3079443,"monotonic":2349864.673579935,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.444","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:48.476","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:48.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.513","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.3087304,"monotonic":2349865.674366354,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.388","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.399","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:49.409","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.419","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.429","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.439","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.448","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.309572,"monotonic":2349866.675207965,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.388","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.400","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.410","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.420","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.429","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.438","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.3104167,"monotonic":2349867.676052502,"hcas":{"mlx5_0":{"port_xmit_data":350026434677072,"port_rcv_data":350255913363563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343026715,"port_rcv_data":1323814958386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.408","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.419","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.430","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.439","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.449","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.458","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.467","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.311244,"monotonic":2349868.676879859,"hcas":{"mlx5_0":{"port_xmit_data":350026434938480,"port_rcv_data":350256002872253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851343294727,"port_rcv_data":1323904473479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.390","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":232.9,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.401","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":262.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.412","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":171.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.422","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":242.99,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.431","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.441","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.450","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.458","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.3120859,"monotonic":2349869.677721671,"hcas":{"mlx5_0":{"port_xmit_data":350026550034229,"port_rcv_data":350256091332071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851458384098,"port_rcv_data":1323992906254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990523631856,"port_rcv_data":350348835031752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1949823102863,"port_rcv_data":1255392880131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.429","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":281.06,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.440","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":292.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.451","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":276.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.461","gpu_util_pct":86.0,"memory_used_mib":71131.0,"power_w":287.02,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.471","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":248.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.484","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":221.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":158.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.505","gpu_util_pct":11.0,"memory_used_mib":71769.0,"power_w":235.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.3128994,"monotonic":2349870.678534994,"hcas":{"mlx5_0":{"port_xmit_data":350026550034229,"port_rcv_data":350256091332071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851458384098,"port_rcv_data":1323992906254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990875610383,"port_rcv_data":350348835748499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950175047073,"port_rcv_data":1255393581661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.393","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":99.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.405","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:54.416","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":115.03,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.426","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":99.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.436","gpu_util_pct":97.0,"memory_used_mib":71769.0,"power_w":290.26,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.446","gpu_util_pct":38.0,"memory_used_mib":71769.0,"power_w":297.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.455","gpu_util_pct":93.0,"memory_used_mib":71769.0,"power_w":265.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.464","gpu_util_pct":12.0,"memory_used_mib":71769.0,"power_w":309.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.3137417,"monotonic":2349871.679377224,"hcas":{"mlx5_0":{"port_xmit_data":350026550034229,"port_rcv_data":350256091332071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851458384098,"port_rcv_data":1323992906254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990875610383,"port_rcv_data":350348835748499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950175047073,"port_rcv_data":1255393581661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.431","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.442","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.453","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.99,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.464","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.474","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":103.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.484","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":96.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.494","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.35,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.503","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":84.93,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.3145437,"monotonic":2349872.680179414,"hcas":{"mlx5_0":{"port_xmit_data":350026550034429,"port_rcv_data":350256091332219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851458384098,"port_rcv_data":1323992906254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990875610383,"port_rcv_data":350348835748499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950175047073,"port_rcv_data":1255393581661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.454","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.463","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.471","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.479","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":76.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.494","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.502","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.64,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.509","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.3153424,"monotonic":2349873.680978278,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.395","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.406","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.417","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.428","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.8,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.465","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.26,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.3161795,"monotonic":2349874.681815409,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.395","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.407","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.417","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.428","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.437","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.447","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.465","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.3170173,"monotonic":2349875.682653318,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.461","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.483","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.492","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.502","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.511","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.520","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.529","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.3177924,"monotonic":2349876.683428464,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.457","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.468","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.478","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.86,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.488","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.497","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.507","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.84,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.525","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.318571,"monotonic":2349877.684206888,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.461","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.482","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.85,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:01.493","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.502","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.512","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.521","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.530","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.19,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.3193457,"monotonic":2349878.68498151,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.416","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.427","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.437","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.46,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.448","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.457","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.467","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.485","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.3201654,"monotonic":2349879.685801156,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.400","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.411","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.421","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:03.431","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.441","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.451","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.460","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.469","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.321002,"monotonic":2349880.68663779,"hcas":{"mlx5_0":{"port_xmit_data":350026690076284,"port_rcv_data":350256091479285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561226622,"port_rcv_data":1323993018319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.424","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.435","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.464","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.474","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.483","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.491","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.321819,"monotonic":2349881.687455108,"hcas":{"mlx5_0":{"port_xmit_data":350026690343745,"port_rcv_data":350256180994182,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851561497539,"port_rcv_data":1324082536564,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349990978453593,"port_rcv_data":350348835861896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950315088533,"port_rcv_data":1255393730849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.446","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":199.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.458","gpu_util_pct":58.0,"memory_used_mib":71131.0,"power_w":201.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.470","gpu_util_pct":63.0,"memory_used_mib":71131.0,"power_w":210.96,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.481","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":194.03,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.491","gpu_util_pct":98.0,"memory_used_mib":71769.0,"power_w":91.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.502","gpu_util_pct":89.0,"memory_used_mib":71769.0,"power_w":80.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.514","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":100.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.523","gpu_util_pct":96.0,"memory_used_mib":71769.0,"power_w":80.06,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.3225982,"monotonic":2349882.68823395,"hcas":{"mlx5_0":{"port_xmit_data":350026805445366,"port_rcv_data":350256268355855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851676589074,"port_rcv_data":1324169852373,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991212658143,"port_rcv_data":350348836305605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950549308456,"port_rcv_data":1255394190225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.404","gpu_util_pct":83.0,"memory_used_mib":71131.0,"power_w":296.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.415","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":288.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.425","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":299.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.436","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":269.9,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.445","gpu_util_pct":52.0,"memory_used_mib":71769.0,"power_w":264.2,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.456","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":265.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.466","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":284.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.476","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":282.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.3234265,"monotonic":2349883.689062386,"hcas":{"mlx5_0":{"port_xmit_data":350026805445366,"port_rcv_data":350256268355855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851676589074,"port_rcv_data":1324169852373,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991328958297,"port_rcv_data":350348836582740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950665592079,"port_rcv_data":1255394479445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.421","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":102.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.432","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":98.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.443","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":107.11,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.454","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.463","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":273.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.473","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":270.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.482","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":256.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.491","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":256.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.3242416,"monotonic":2349884.68987758,"hcas":{"mlx5_0":{"port_xmit_data":350026805445366,"port_rcv_data":350256268355855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851676589074,"port_rcv_data":1324169852373,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991328958297,"port_rcv_data":350348836582740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950665592079,"port_rcv_data":1255394479445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.420","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.431","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.442","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.46,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.452","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.463","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":106.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.472","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":100.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.481","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":84.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.490","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":88.13,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.3250732,"monotonic":2349885.690708464,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.471","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.483","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:09.494","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.86,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:09.505","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.515","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.524","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.534","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.7,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.543","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.3258452,"monotonic":2349886.69148123,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.426","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.438","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.449","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.470","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.479","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.489","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.8,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.498","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.31,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.3266654,"monotonic":2349887.692301214,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.428","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.440","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.450","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.46,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.470","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.479","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.489","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.497","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.3274848,"monotonic":2349888.693120755,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.464","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.476","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.486","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.497","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.506","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.525","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.534","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.3282707,"monotonic":2349889.693906509,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.465","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.476","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:13.487","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.87,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.497","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.507","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.525","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.534","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.3290672,"monotonic":2349890.694702772,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.449","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.471","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.88,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.481","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.490","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.86,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.517","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.48,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.329873,"monotonic":2349891.695508771,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.433","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.444","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.46,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.475","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.485","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.494","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.503","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.3306923,"monotonic":2349892.696328022,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.473","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:16.484","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:16.495","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.505","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.514","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.523","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.532","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.541","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.331473,"monotonic":2349893.697109092,"hcas":{"mlx5_0":{"port_xmit_data":350026943393712,"port_rcv_data":350256268504196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851777933622,"port_rcv_data":1324169963306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.410","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.422","gpu_util_pct":61.0,"memory_used_mib":71131.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.434","gpu_util_pct":77.0,"memory_used_mib":71131.0,"power_w":82.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.456","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.465","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.474","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.482","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.3323023,"monotonic":2349894.697938533,"hcas":{"mlx5_0":{"port_xmit_data":350027058743944,"port_rcv_data":350256443834304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851893267745,"port_rcv_data":1324345247103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991430305587,"port_rcv_data":350348836694695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1950803541691,"port_rcv_data":1255394626742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.431","gpu_util_pct":56.0,"memory_used_mib":71131.0,"power_w":185.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.443","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":211.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:18.455","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":308.06,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.466","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":269.85,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.477","gpu_util_pct":95.0,"memory_used_mib":71769.0,"power_w":96.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.487","gpu_util_pct":97.0,"memory_used_mib":71769.0,"power_w":99.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.497","gpu_util_pct":89.0,"memory_used_mib":71769.0,"power_w":112.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.508","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":105.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.3331091,"monotonic":2349895.69874525,"hcas":{"mlx5_0":{"port_xmit_data":350027058743944,"port_rcv_data":350256443834304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851893267745,"port_rcv_data":1324345247103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991664515857,"port_rcv_data":350348837144139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951037765383,"port_rcv_data":1255395089640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.432","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":264.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.444","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":251.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.454","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":258.31,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.464","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":243.76,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.474","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":306.28,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.484","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":278.64,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.493","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":271.89,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.503","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":282.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.3339245,"monotonic":2349896.699560665,"hcas":{"mlx5_0":{"port_xmit_data":350027058743944,"port_rcv_data":350256443834304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851893267745,"port_rcv_data":1324345247103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991778803031,"port_rcv_data":350348837441122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951152018660,"port_rcv_data":1255395381873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.437","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":93.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:20.448","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:20.458","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.469","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":106.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.479","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":223.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.488","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":234.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.497","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":223.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.506","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":228.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.3343868,"monotonic":2349897.700022824,"hcas":{"mlx5_0":{"port_xmit_data":350027058743944,"port_rcv_data":350256443834304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851893267745,"port_rcv_data":1324345247103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991778803031,"port_rcv_data":350348837441122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951152018660,"port_rcv_data":1255395381873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.413","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.424","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.435","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.51,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.454","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.463","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.472","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":101.63,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.481","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":111.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.3352196,"monotonic":2349898.70085557,"hcas":{"mlx5_0":{"port_xmit_data":350027194598131,"port_rcv_data":350256443983684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851993118749,"port_rcv_data":1324345356889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991878654688,"port_rcv_data":350348837553138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951287871992,"port_rcv_data":1255395526441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.477","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.487","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.497","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.507","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.74,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.525","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.3360045,"monotonic":2349899.701640569,"hcas":{"mlx5_0":{"port_xmit_data":350027194598131,"port_rcv_data":350256443983684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851993118749,"port_rcv_data":1324345356889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991878654688,"port_rcv_data":350348837553138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951287871992,"port_rcv_data":1255395526441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.449","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.471","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.481","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.490","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.0,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.517","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.3367991,"monotonic":2349900.702435082,"hcas":{"mlx5_0":{"port_xmit_data":350027194598131,"port_rcv_data":350256443983684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851993118749,"port_rcv_data":1324345356889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991878654688,"port_rcv_data":350348837553138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951287871992,"port_rcv_data":1255395526441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.432","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.443","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.454","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.46,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.464","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.473","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.482","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.491","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.500","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.3376095,"monotonic":2349901.703245498,"hcas":{"mlx5_0":{"port_xmit_data":350027194859206,"port_rcv_data":350256533492212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1851993378814,"port_rcv_data":1324434864328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991878654688,"port_rcv_data":350348837553138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951287871992,"port_rcv_data":1255395526441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.454","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":149.66,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.466","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":160.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.478","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":179.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.491","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":166.86,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.501","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.510","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.519","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.94,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.528","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.338383,"monotonic":2349902.70401867,"hcas":{"mlx5_0":{"port_xmit_data":350027309947800,"port_rcv_data":350256619155905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852108440910,"port_rcv_data":1324520482006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349991878654688,"port_rcv_data":350348837553138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951287871992,"port_rcv_data":1255395526441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.439","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":282.37,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.450","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":285.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.461","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":293.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.471","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":289.35,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.481","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":197.13,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.490","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":200.32,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.500","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":191.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.509","gpu_util_pct":96.0,"memory_used_mib":71769.0,"power_w":206.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.339167,"monotonic":2349903.704803065,"hcas":{"mlx5_0":{"port_xmit_data":350027309947800,"port_rcv_data":350256619155905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852108440910,"port_rcv_data":1324520482006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992112870646,"port_rcv_data":350348838008999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951522095262,"port_rcv_data":1255395989661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.421","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":153.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.432","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":150.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.443","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":152.34,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.454","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":155.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.464","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":278.52,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.474","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":277.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.483","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":285.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.492","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":304.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.339974,"monotonic":2349904.705609894,"hcas":{"mlx5_0":{"port_xmit_data":350027309947800,"port_rcv_data":350256619155905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852108440910,"port_rcv_data":1324520482006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992226947334,"port_rcv_data":350348838304234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951636149769,"port_rcv_data":1255396283629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.437","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":112.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.448","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":96.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.458","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":94.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:28.468","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.478","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":110.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":112.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.493","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.501","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":111.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.3407762,"monotonic":2349905.706412121,"hcas":{"mlx5_0":{"port_xmit_data":350027309947800,"port_rcv_data":350256619155905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852108440910,"port_rcv_data":1324520482006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992226947334,"port_rcv_data":350348838304234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951636149769,"port_rcv_data":1255396283629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.436","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.448","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.458","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.64,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.468","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.478","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":86.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.487","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":85.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.99,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.3415682,"monotonic":2349906.707204224,"hcas":{"mlx5_0":{"port_xmit_data":350027309948065,"port_rcv_data":350256619156118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852108440910,"port_rcv_data":1324520482006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992226947334,"port_rcv_data":350348838304234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951636149769,"port_rcv_data":1255396283629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.440","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.451","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.462","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.99,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.482","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.491","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.500","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.17,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.509","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.3423555,"monotonic":2349907.707991314,"hcas":{"mlx5_0":{"port_xmit_data":350027445797243,"port_rcv_data":350256619301488,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852208287298,"port_rcv_data":1324520591405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992226951950,"port_rcv_data":350348838305474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951636154513,"port_rcv_data":1255396284869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.422","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.434","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.97,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.465","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.475","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.484","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.493","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.3424017,"monotonic":2349908.708037346,"hcas":{"mlx5_0":{"port_xmit_data":350027446047452,"port_rcv_data":350256708795428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852208551355,"port_rcv_data":1324610099353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992326798927,"port_rcv_data":350348838415321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951772003101,"port_rcv_data":1255396427146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.443","gpu_util_pct":61.0,"memory_used_mib":71131.0,"power_w":123.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.455","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":126.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:32.468","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":124.33,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:32.478","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":125.38,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.488","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.497","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.506","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.515","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.3432112,"monotonic":2349909.708846945,"hcas":{"mlx5_0":{"port_xmit_data":350027561145831,"port_rcv_data":350256797099268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852323639744,"port_rcv_data":1324698362960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992326798927,"port_rcv_data":350348838415321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1951772003101,"port_rcv_data":1255396427146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.450","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":268.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.462","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":257.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.473","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":281.33,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.483","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":265.83,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.493","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":159.11,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.503","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":171.51,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.514","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":177.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.523","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":177.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.3440018,"monotonic":2349910.709637815,"hcas":{"mlx5_0":{"port_xmit_data":350027561145831,"port_rcv_data":350256797099268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852323639744,"port_rcv_data":1324698362960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992560992297,"port_rcv_data":350348838847586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952006217401,"port_rcv_data":1255396880202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.424","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":213.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.436","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":231.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.446","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":245.91,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.456","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":227.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.466","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":255.76,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.476","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":286.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.486","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":260.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.496","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":297.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.3448362,"monotonic":2349911.710472418,"hcas":{"mlx5_0":{"port_xmit_data":350027561145831,"port_rcv_data":350256797099268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852323639744,"port_rcv_data":1324698362960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992678555032,"port_rcv_data":350348839118075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952123722361,"port_rcv_data":1255397133823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.424","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.436","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":110.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:35.446","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":113.53,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:35.457","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.467","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":202.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.476","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":210.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":204.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":201.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.3456652,"monotonic":2349912.711301164,"hcas":{"mlx5_0":{"port_xmit_data":350027561145831,"port_rcv_data":350256797099268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852323639744,"port_rcv_data":1324698362960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992678555032,"port_rcv_data":350348839118075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952123722361,"port_rcv_data":1255397133823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.444","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.466","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.5,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.476","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":106.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.505","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":101.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.514","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":103.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.3464801,"monotonic":2349913.712116294,"hcas":{"mlx5_0":{"port_xmit_data":350027561145831,"port_rcv_data":350256797099268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852323639744,"port_rcv_data":1324698362960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992678555032,"port_rcv_data":350348839118075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952123722361,"port_rcv_data":1255397133823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.425","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.436","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:37.447","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.41,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.457","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.466","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.475","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.484","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.493","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.347312,"monotonic":2349914.712948024,"hcas":{"mlx5_0":{"port_xmit_data":350027561146096,"port_rcv_data":350256797099481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852323639744,"port_rcv_data":1324698362960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992678555032,"port_rcv_data":350348839118075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952123722361,"port_rcv_data":1255397133823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.426","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.437","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.448","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.48,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.458","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.468","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.477","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.486","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.3481433,"monotonic":2349915.713779147,"hcas":{"mlx5_0":{"port_xmit_data":350027561146096,"port_rcv_data":350256797099481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852323639744,"port_rcv_data":1324698362960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992678555032,"port_rcv_data":350348839118075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952123722361,"port_rcv_data":1255397133823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.427","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.438","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.449","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.13,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.459","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.469","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.478","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.487","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.3490105,"monotonic":2349916.714645887,"hcas":{"mlx5_0":{"port_xmit_data":350027701455872,"port_rcv_data":350256886762338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852426705474,"port_rcv_data":1324787943513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349992781397704,"port_rcv_data":350348839230278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952263763821,"port_rcv_data":1255397283150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.455","gpu_util_pct":51.0,"memory_used_mib":71131.0,"power_w":230.93,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.465","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":239.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.476","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":249.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.486","gpu_util_pct":38.0,"memory_used_mib":71131.0,"power_w":221.8,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.495","gpu_util_pct":39.0,"memory_used_mib":71769.0,"power_w":80.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.505","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.514","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.11,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.524","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.28,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.3498242,"monotonic":2349917.715459929,"hcas":{"mlx5_0":{"port_xmit_data":350027816514013,"port_rcv_data":350256971547225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852541764012,"port_rcv_data":1324872703452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993015615354,"port_rcv_data":350348839686534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952497987904,"port_rcv_data":1255397745865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.449","gpu_util_pct":96.0,"memory_used_mib":71131.0,"power_w":254.47,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.461","gpu_util_pct":4.0,"memory_used_mib":71131.0,"power_w":286.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":301.17,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.482","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":269.07,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.492","gpu_util_pct":5.0,"memory_used_mib":71769.0,"power_w":295.12,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.503","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":244.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.514","gpu_util_pct":56.0,"memory_used_mib":71769.0,"power_w":262.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.524","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":282.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.3503926,"monotonic":2349918.716028205,"hcas":{"mlx5_0":{"port_xmit_data":350027816514013,"port_rcv_data":350256971547225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852541764012,"port_rcv_data":1324872703452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993128530494,"port_rcv_data":350348839949060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952610901077,"port_rcv_data":1255398035580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.470","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":87.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:42.482","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:42.493","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.68,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.503","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":97.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.513","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":255.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.523","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":252.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.533","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":250.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.542","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":285.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.3511941,"monotonic":2349919.716829778,"hcas":{"mlx5_0":{"port_xmit_data":350027816514013,"port_rcv_data":350256971547225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852541764012,"port_rcv_data":1324872703452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993128530494,"port_rcv_data":350348839949060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952610901077,"port_rcv_data":1255398035580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.448","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.458","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.469","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.479","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.489","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":104.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":91.38,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.517","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":106.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.3520222,"monotonic":2349920.717658115,"hcas":{"mlx5_0":{"port_xmit_data":350027816514278,"port_rcv_data":350256971547438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852541764012,"port_rcv_data":1324872703452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993128530494,"port_rcv_data":350348839949060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952610901077,"port_rcv_data":1255398035580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.470","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.482","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.493","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.35,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.504","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.514","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.524","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.533","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.99,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.542","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.3528368,"monotonic":2349921.718472517,"hcas":{"mlx5_0":{"port_xmit_data":350027951318513,"port_rcv_data":350256971688128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852640861380,"port_rcv_data":1324872809612,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993128532830,"port_rcv_data":350348839950052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952610906801,"port_rcv_data":1255398036944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:45.452","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:45.464","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:45.475","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.37,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.486","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.496","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.506","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.515","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.43,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.524","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.353635,"monotonic":2349922.719270844,"hcas":{"mlx5_0":{"port_xmit_data":350027951321105,"port_rcv_data":350256971689120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852640864760,"port_rcv_data":1324872810728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993227634049,"port_rcv_data":350348840059018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952745709369,"port_rcv_data":1255398179854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.453","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.465","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.476","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.23,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.487","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.497","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.507","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.29,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.526","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.3543997,"monotonic":2349923.720035192,"hcas":{"mlx5_0":{"port_xmit_data":350027951566902,"port_rcv_data":350257061182633,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852641119317,"port_rcv_data":1324962312675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993227634049,"port_rcv_data":350348840059018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952745709369,"port_rcv_data":1255398179854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.454","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":176.66,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.466","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":181.7,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.478","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":185.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.490","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":179.71,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.501","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.511","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.520","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.16,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.529","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.355229,"monotonic":2349924.720864438,"hcas":{"mlx5_0":{"port_xmit_data":350028066700731,"port_rcv_data":350257149951827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852756236444,"port_rcv_data":1325051059445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993227634049,"port_rcv_data":350348840059018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952745709369,"port_rcv_data":1255398179854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.477","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":262.17,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.489","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":252.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.501","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":280.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.513","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":255.5,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.525","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":192.36,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.534","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":214.09,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.544","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":204.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.553","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":208.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.3560286,"monotonic":2349925.721664153,"hcas":{"mlx5_0":{"port_xmit_data":350028066944804,"port_rcv_data":350257239441136,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852756470895,"port_rcv_data":1325140540527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993461840451,"port_rcv_data":350348840504394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1952979923864,"port_rcv_data":1255398633338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.456","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":323.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.468","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":323.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.479","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":321.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.490","gpu_util_pct":58.0,"memory_used_mib":71131.0,"power_w":326.8,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.500","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":281.86,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.510","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":251.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.519","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":264.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.530","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":268.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.3568552,"monotonic":2349926.722490566,"hcas":{"mlx5_0":{"port_xmit_data":350028182055768,"port_rcv_data":350257327120180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852871593036,"port_rcv_data":1325228213931,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993579986657,"port_rcv_data":350348840784391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953098062100,"port_rcv_data":1255398913615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.460","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":262.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.473","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":253.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.485","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":261.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.497","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":264.89,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":261.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.518","gpu_util_pct":40.0,"memory_used_mib":71769.0,"power_w":286.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.528","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":266.33,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.537","gpu_util_pct":13.0,"memory_used_mib":71769.0,"power_w":313.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.3576653,"monotonic":2349927.723300933,"hcas":{"mlx5_0":{"port_xmit_data":350028182055768,"port_rcv_data":350257327120180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852871593036,"port_rcv_data":1325228213931,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993814187989,"port_rcv_data":350348841224185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953332262262,"port_rcv_data":1255399352471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.457","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":120.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.468","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":127.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.480","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":141.64,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.490","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":109.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.500","gpu_util_pct":61.0,"memory_used_mib":71769.0,"power_w":285.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.510","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":274.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.520","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":282.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.529","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":300.23,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.358487,"monotonic":2349928.724122599,"hcas":{"mlx5_0":{"port_xmit_data":350028182055768,"port_rcv_data":350257327120180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852871593036,"port_rcv_data":1325228213931,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993930910178,"port_rcv_data":350348841508934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953448980919,"port_rcv_data":1255399650118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:52.438","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:52.450","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.461","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.472","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.482","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":107.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.492","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":103.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.501","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":110.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.510","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":102.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.3593347,"monotonic":2349929.724970067,"hcas":{"mlx5_0":{"port_xmit_data":350028182055768,"port_rcv_data":350257327120180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852871593036,"port_rcv_data":1325228213931,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993930910178,"port_rcv_data":350348841508934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953448980919,"port_rcv_data":1255399650118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.456","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.467","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.479","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.39,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.489","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.517","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.526","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.3601825,"monotonic":2349930.725818046,"hcas":{"mlx5_0":{"port_xmit_data":350028264755333,"port_rcv_data":350257327211017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852935608915,"port_rcv_data":1325228284053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349993930913348,"port_rcv_data":350348841509740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953448983761,"port_rcv_data":1255399650878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.446","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.460","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.471","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.62,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.481","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.491","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.500","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.509","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.517","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.3610218,"monotonic":2349931.726656852,"hcas":{"mlx5_0":{"port_xmit_data":350028322098608,"port_rcv_data":350257327271057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852974435560,"port_rcv_data":1325228327174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994033752850,"port_rcv_data":350348841622377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953589022379,"port_rcv_data":1255399798143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.463","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.475","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.486","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.74,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.496","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.506","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.525","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.534","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.3618312,"monotonic":2349932.727466565,"hcas":{"mlx5_0":{"port_xmit_data":350028322098608,"port_rcv_data":350257327271057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1852974435560,"port_rcv_data":1325228327174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994033753323,"port_rcv_data":350348841622902,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953589022379,"port_rcv_data":1255399798143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:56.497","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.506","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.515","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.53,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.524","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.533","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.541","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.549","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.557","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.3623943,"monotonic":2349933.728030007,"hcas":{"mlx5_0":{"port_xmit_data":350028442915166,"port_rcv_data":350257416919981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853095239086,"port_rcv_data":1325317964916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994154298365,"port_rcv_data":350348841754373,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953709566915,"port_rcv_data":1255399927444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.467","gpu_util_pct":39.0,"memory_used_mib":71131.0,"power_w":242.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.479","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":255.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.491","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":268.43,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.502","gpu_util_pct":32.0,"memory_used_mib":71131.0,"power_w":234.91,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.515","gpu_util_pct":29.0,"memory_used_mib":71769.0,"power_w":80.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.525","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.535","gpu_util_pct":77.0,"memory_used_mib":71769.0,"power_w":99.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.545","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.3631935,"monotonic":2349934.728829254,"hcas":{"mlx5_0":{"port_xmit_data":350028558007715,"port_rcv_data":350257505714047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853210328417,"port_rcv_data":1325406730669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994388503946,"port_rcv_data":350348842198359,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1953943778503,"port_rcv_data":1255400377975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:58.483","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":248.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.496","gpu_util_pct":61.0,"memory_used_mib":71131.0,"power_w":293.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.507","gpu_util_pct":58.0,"memory_used_mib":71131.0,"power_w":299.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.518","gpu_util_pct":13.0,"memory_used_mib":71131.0,"power_w":266.96,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.528","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":277.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.539","gpu_util_pct":64.0,"memory_used_mib":71769.0,"power_w":279.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.550","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":287.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.560","gpu_util_pct":89.0,"memory_used_mib":71769.0,"power_w":274.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.363983,"monotonic":2349935.729618252,"hcas":{"mlx5_0":{"port_xmit_data":350028558007715,"port_rcv_data":350257505714047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853210328417,"port_rcv_data":1325406730669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994506734166,"port_rcv_data":350348842472241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954061977783,"port_rcv_data":1255400654000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.462","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":107.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.474","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":100.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.485","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":105.78,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.496","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.87,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.506","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":266.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.515","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":286.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.525","gpu_util_pct":28.0,"memory_used_mib":71769.0,"power_w":238.13,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.534","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":284.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.3647923,"monotonic":2349936.730428008,"hcas":{"mlx5_0":{"port_xmit_data":350028558007715,"port_rcv_data":350257505714047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853210328417,"port_rcv_data":1325406730669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994506734166,"port_rcv_data":350348842472241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954061977783,"port_rcv_data":1255400654000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.465","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.477","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.488","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.498","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":105.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.518","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.527","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":87.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.537","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":98.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.3655987,"monotonic":2349937.731234403,"hcas":{"mlx5_0":{"port_xmit_data":350028558007715,"port_rcv_data":350257505714047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853210328417,"port_rcv_data":1325406730669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994506734166,"port_rcv_data":350348842472241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954061977783,"port_rcv_data":1255400654000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:01.481","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.492","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.503","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.514","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.524","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.533","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.543","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.552","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.3664117,"monotonic":2349938.732047468,"hcas":{"mlx5_0":{"port_xmit_data":350028558007715,"port_rcv_data":350257505714047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853210328417,"port_rcv_data":1325406730669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994506734166,"port_rcv_data":350348842472241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954061977783,"port_rcv_data":1255400654000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.445","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.457","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.468","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.478","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.488","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.497","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.506","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.19,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.515","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.3672533,"monotonic":2349939.732888908,"hcas":{"mlx5_0":{"port_xmit_data":350028558269565,"port_rcv_data":350257595223405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853210600913,"port_rcv_data":1325496250508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994506734166,"port_rcv_data":350348842472241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954061977783,"port_rcv_data":1255400654000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.492","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.504","gpu_util_pct":60.0,"memory_used_mib":71131.0,"power_w":98.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.516","gpu_util_pct":64.0,"memory_used_mib":71131.0,"power_w":100.87,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.528","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":102.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.539","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.548","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.558","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.567","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.3680375,"monotonic":2349940.733673188,"hcas":{"mlx5_0":{"port_xmit_data":350028813407171,"port_rcv_data":350257683506764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853428546188,"port_rcv_data":1325584491629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994506738782,"port_rcv_data":350348842473481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954061982527,"port_rcv_data":1255400655240,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.486","gpu_util_pct":87.0,"memory_used_mib":71131.0,"power_w":284.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.498","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":275.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.510","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":280.35,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.521","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":285.63,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.533","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":134.45,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.544","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":138.09,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.554","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":139.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.564","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":135.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.3688228,"monotonic":2349941.734458578,"hcas":{"mlx5_0":{"port_xmit_data":350028813411915,"port_rcv_data":350257683508004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853428550804,"port_rcv_data":1325584492869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994843826628,"port_rcv_data":350348843071874,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954436244820,"port_rcv_data":1255401268203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.475","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":239.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.487","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":232.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.498","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":259.18,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.510","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":247.35,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.520","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":256.09,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.530","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":290.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.541","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":276.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.551","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":292.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.3696232,"monotonic":2349942.735258825,"hcas":{"mlx5_0":{"port_xmit_data":350028813411915,"port_rcv_data":350257683508004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853428550804,"port_rcv_data":1325584492869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994961173724,"port_rcv_data":350348843359628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954553577561,"port_rcv_data":1255401554465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:06.487","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.499","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.510","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.12,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.521","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.531","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":206.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.541","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":191.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.551","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":202.16,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.560","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":183.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.3704185,"monotonic":2349943.736054249,"hcas":{"mlx5_0":{"port_xmit_data":350028813411915,"port_rcv_data":350257683508004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853428550804,"port_rcv_data":1325584492869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994961173724,"port_rcv_data":350348843359628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954553577561,"port_rcv_data":1255401554465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.450","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.462","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.474","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.03,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.484","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.494","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.514","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.523","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.371252,"monotonic":2349944.736887674,"hcas":{"mlx5_0":{"port_xmit_data":350028813411915,"port_rcv_data":350257683508004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853428550804,"port_rcv_data":1325584492869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994961173724,"port_rcv_data":350348843359628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954553577561,"port_rcv_data":1255401554465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.450","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.462","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.473","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.484","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.494","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.513","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.92,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.523","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.3720875,"monotonic":2349945.737723461,"hcas":{"mlx5_0":{"port_xmit_data":350028813411915,"port_rcv_data":350257683508004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853428550804,"port_rcv_data":1325584492869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994961173724,"port_rcv_data":350348843359628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954553577561,"port_rcv_data":1255401554465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.451","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.463","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.474","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.39,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.485","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.495","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.505","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.515","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.524","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.372935,"monotonic":2349946.738570784,"hcas":{"mlx5_0":{"port_xmit_data":350028928728077,"port_rcv_data":350257860633092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853543912954,"port_rcv_data":1325761619023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349994961173724,"port_rcv_data":350348843359628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954553577561,"port_rcv_data":1255401554465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.475","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":180.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.486","gpu_util_pct":54.0,"memory_used_mib":71131.0,"power_w":179.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.495","gpu_util_pct":59.0,"memory_used_mib":71131.0,"power_w":195.13,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.505","gpu_util_pct":99.0,"memory_used_mib":71131.0,"power_w":175.51,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.513","gpu_util_pct":98.0,"memory_used_mib":71769.0,"power_w":92.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.522","gpu_util_pct":89.0,"memory_used_mib":71769.0,"power_w":80.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.533","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":99.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.544","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":82.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.3737628,"monotonic":2349947.739398379,"hcas":{"mlx5_0":{"port_xmit_data":350029049269409,"port_rcv_data":350257860764016,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853664452810,"port_rcv_data":1325761749161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995195356698,"port_rcv_data":350348843777097,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1954787765618,"port_rcv_data":1255401977388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.477","gpu_util_pct":91.0,"memory_used_mib":71131.0,"power_w":300.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.489","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":289.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.501","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":279.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.512","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":274.51,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.523","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":278.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.534","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":260.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.545","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":280.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.556","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":264.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.374401,"monotonic":2349948.740036639,"hcas":{"mlx5_0":{"port_xmit_data":350029049274089,"port_rcv_data":350257860765256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853664457490,"port_rcv_data":1325761750401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995432633196,"port_rcv_data":350348844216487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955025001760,"port_rcv_data":1255402419624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.474","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.486","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.497","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.51,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.508","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.518","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":253.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.528","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":260.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.538","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":232.52,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.547","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":269.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.3752294,"monotonic":2349949.740864951,"hcas":{"mlx5_0":{"port_xmit_data":350029049274089,"port_rcv_data":350257860765256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853664457490,"port_rcv_data":1325761750401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995432633734,"port_rcv_data":350348844217077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955025001760,"port_rcv_data":1255402419624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:13.474","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.485","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.497","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.508","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.518","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.528","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.537","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":103.38,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.546","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":112.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.3760507,"monotonic":2349950.741686362,"hcas":{"mlx5_0":{"port_xmit_data":350029049274089,"port_rcv_data":350257860765256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853664457490,"port_rcv_data":1325761750401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995432633734,"port_rcv_data":350348844217077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955025001760,"port_rcv_data":1255402419624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.455","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.467","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.478","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.71,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.489","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.499","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.508","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.517","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.526","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.3769019,"monotonic":2349951.742537539,"hcas":{"mlx5_0":{"port_xmit_data":350029049274354,"port_rcv_data":350257860765469,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853664457490,"port_rcv_data":1325761750401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995432633734,"port_rcv_data":350348844217077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955025001760,"port_rcv_data":1255402419624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.521","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.533","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.544","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.67,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.555","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.565","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.575","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.586","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.596","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.377677,"monotonic":2349952.743312694,"hcas":{"mlx5_0":{"port_xmit_data":350029049274354,"port_rcv_data":350257860765469,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853664457490,"port_rcv_data":1325761750401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995432633734,"port_rcv_data":350348844217077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955025001760,"port_rcv_data":1255402419624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:16.457","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.469","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.480","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.8,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.490","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.501","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.510","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.519","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.06,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.528","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.37853,"monotonic":2349953.744165995,"hcas":{"mlx5_0":{"port_xmit_data":350029170043471,"port_rcv_data":350257950369851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853785244460,"port_rcv_data":1325851372625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995432638627,"port_rcv_data":350348844218582,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955025006440,"port_rcv_data":1255402420864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.481","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":182.9,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.493","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":168.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.505","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":189.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.517","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":202.64,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.529","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.538","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.547","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.31,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.557","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.3793578,"monotonic":2349954.744993566,"hcas":{"mlx5_0":{"port_xmit_data":350029285121778,"port_rcv_data":350258039926283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1853900333821,"port_rcv_data":1325940938825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995553178483,"port_rcv_data":350348844349077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955145546296,"port_rcv_data":1255402549421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.498","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":300.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.510","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":296.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.522","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":295.05,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.533","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":286.87,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.544","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":232.57,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.553","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":209.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.563","gpu_util_pct":54.0,"memory_used_mib":71769.0,"power_w":238.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.572","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":241.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.3801613,"monotonic":2349955.745797095,"hcas":{"mlx5_0":{"port_xmit_data":350029400076700,"port_rcv_data":350258070244966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854015289493,"port_rcv_data":1325971242395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349995787393436,"port_rcv_data":350348844802589,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955379762351,"port_rcv_data":1255403003922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.480","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":252.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.492","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":246.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.504","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":242.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.515","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":212.52,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.525","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":291.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.536","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":283.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.547","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":286.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.557","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":264.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.3809729,"monotonic":2349956.746608455,"hcas":{"mlx5_0":{"port_xmit_data":350029400076700,"port_rcv_data":350258070244966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854015289493,"port_rcv_data":1325971242395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996021590425,"port_rcv_data":350348845242819,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955613958224,"port_rcv_data":1255403444964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.461","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.474","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.484","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.494","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.0,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.504","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":216.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.513","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":221.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.523","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":215.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.532","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":224.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.38182,"monotonic":2349957.747455386,"hcas":{"mlx5_0":{"port_xmit_data":350029400076700,"port_rcv_data":350258070244966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854015289493,"port_rcv_data":1325971242395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996061862223,"port_rcv_data":350348845343179,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955654221181,"port_rcv_data":1255403557323,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:21.480","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.492","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.504","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.2,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.514","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.85,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.524","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.534","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.543","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.7,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.552","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.382658,"monotonic":2349958.748293569,"hcas":{"mlx5_0":{"port_xmit_data":350029447024559,"port_rcv_data":350258070302096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854062236037,"port_rcv_data":1325971298629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996061867357,"port_rcv_data":350348845344964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955654225861,"port_rcv_data":1255403558563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.523","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.535","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.546","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.28,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.557","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.568","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.578","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.588","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.597","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.3834405,"monotonic":2349959.749076266,"hcas":{"mlx5_0":{"port_xmit_data":350029447029239,"port_rcv_data":350258070303336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854062240717,"port_rcv_data":1325971299869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996108814573,"port_rcv_data":350348845400773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955701172405,"port_rcv_data":1255403613712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.498","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.510","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.521","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.81,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.531","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.541","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.551","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.560","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.569","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.3842509,"monotonic":2349960.749886606,"hcas":{"mlx5_0":{"port_xmit_data":350029447029239,"port_rcv_data":350258070303336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854062240717,"port_rcv_data":1325971299869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996108814573,"port_rcv_data":350348845400773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955701172405,"port_rcv_data":1255403613712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.481","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.491","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.499","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.507","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.516","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.525","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.533","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.92,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.541","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.3850951,"monotonic":2349961.750730737,"hcas":{"mlx5_0":{"port_xmit_data":350029447297965,"port_rcv_data":350258159819111,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854062516907,"port_rcv_data":1326060823216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996108814573,"port_rcv_data":350348845400773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955701172405,"port_rcv_data":1255403613712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.491","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":111.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.503","gpu_util_pct":99.0,"memory_used_mib":71131.0,"power_w":108.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.515","gpu_util_pct":99.0,"memory_used_mib":71131.0,"power_w":120.4,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.527","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":126.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.538","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.548","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.557","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.566","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.3859084,"monotonic":2349962.751544332,"hcas":{"mlx5_0":{"port_xmit_data":350029562375708,"port_rcv_data":350258244342067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854177595424,"port_rcv_data":1326145340497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996108814573,"port_rcv_data":350348845400773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955701172405,"port_rcv_data":1255403613712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.489","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":286.87,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.501","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":283.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.512","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":294.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.523","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":299.98,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.534","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":171.83,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.545","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":187.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.556","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":179.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.566","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":202.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.3867292,"monotonic":2349963.752364798,"hcas":{"mlx5_0":{"port_xmit_data":350029562375708,"port_rcv_data":350258244342067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854177595424,"port_rcv_data":1326145340497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996343031993,"port_rcv_data":350348845856515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1955935398925,"port_rcv_data":1255404079298,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.487","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":219.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.499","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":198.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.510","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":212.53,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.522","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":183.16,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.532","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":277.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.542","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":279.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.553","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":291.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.562","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":301.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.3875477,"monotonic":2349964.753183402,"hcas":{"mlx5_0":{"port_xmit_data":350029562375708,"port_rcv_data":350258244342067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854177595424,"port_rcv_data":1326145340497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996455552499,"port_rcv_data":350348846094242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956047936581,"port_rcv_data":1255404342316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.485","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.497","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":96.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.508","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":95.53,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.518","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.41,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.528","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":161.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.538","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":132.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.547","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":140.38,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.556","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":173.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.3883736,"monotonic":2349965.75400935,"hcas":{"mlx5_0":{"port_xmit_data":350029562375708,"port_rcv_data":350258244342067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854177595424,"port_rcv_data":1326145340497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996455552499,"port_rcv_data":350348846094242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956047936581,"port_rcv_data":1255404342316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.484","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.496","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.507","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.518","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.528","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":84.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.538","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":85.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.547","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":112.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.557","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":106.31,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.389212,"monotonic":2349966.754847529,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.495","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.506","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.518","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.63,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.528","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.538","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.548","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.557","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.566","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.3900301,"monotonic":2349967.755665692,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.494","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.506","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.517","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.72,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.528","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.539","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.548","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.558","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.94,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.567","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.3908434,"monotonic":2349968.756479063,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.514","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.526","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.538","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.548","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.558","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.568","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.578","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.391668,"monotonic":2349969.757303625,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.527","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.539","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.551","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.48,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.562","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.572","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.582","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.591","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.600","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.3924613,"monotonic":2349970.758096933,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.542","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.554","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.565","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.54,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.576","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.596","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.606","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.26,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.615","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.3932319,"monotonic":2349971.758867658,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.541","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.565","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.576","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.586","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.596","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.606","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.615","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.3940082,"monotonic":2349972.759643546,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.514","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.525","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.537","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.37,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.547","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.557","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.567","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.576","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.585","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.3943908,"monotonic":2349973.760026737,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.492","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.504","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.515","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.49,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.526","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.536","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.546","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.555","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.92,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.564","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.3952153,"monotonic":2349974.760851293,"hcas":{"mlx5_0":{"port_xmit_data":350029678433594,"port_rcv_data":350258244467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293652488,"port_rcv_data":1326145465892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.492","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.504","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.515","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.39,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.526","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.535","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.545","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.555","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.564","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.3960404,"monotonic":2349975.761676225,"hcas":{"mlx5_0":{"port_xmit_data":350029678689417,"port_rcv_data":350258333970758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854293924845,"port_rcv_data":1326234985437,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996571610282,"port_rcv_data":350348846221513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956163993645,"port_rcv_data":1255404468579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.522","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":163.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.533","gpu_util_pct":3.0,"memory_used_mib":71131.0,"power_w":162.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.542","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":165.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.552","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":160.63,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.561","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.569","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":93.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.577","gpu_util_pct":80.0,"memory_used_mib":71769.0,"power_w":81.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":107.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.3968384,"monotonic":2349976.76247419,"hcas":{"mlx5_0":{"port_xmit_data":350029793784845,"port_rcv_data":350258420745414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854409026418,"port_rcv_data":1326321739092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996805836834,"port_rcv_data":350348846685969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956398205995,"port_rcv_data":1255404918988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.511","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":257.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.522","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":266.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.532","gpu_util_pct":95.0,"memory_used_mib":71131.0,"power_w":313.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.542","gpu_util_pct":53.0,"memory_used_mib":71131.0,"power_w":291.85,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.552","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":172.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.562","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":276.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.572","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":271.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.583","gpu_util_pct":45.0,"memory_used_mib":71769.0,"power_w":266.86,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.397638,"monotonic":2349977.763273552,"hcas":{"mlx5_0":{"port_xmit_data":350029793784845,"port_rcv_data":350258420745414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854409026418,"port_rcv_data":1326321739092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996921342960,"port_rcv_data":350348846937280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956513691553,"port_rcv_data":1255405175624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.503","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.514","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.525","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.85,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.536","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.546","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":275.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.556","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":243.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.566","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":270.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.575","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":251.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.3983932,"monotonic":2349978.76402873,"hcas":{"mlx5_0":{"port_xmit_data":350029793784845,"port_rcv_data":350258420745414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854409026418,"port_rcv_data":1326321739092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349996921342960,"port_rcv_data":350348846937280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956513691553,"port_rcv_data":1255405175624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.478","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.490","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.501","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.511","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.521","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.531","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":105.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.540","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":98.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.549","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":102.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.3992298,"monotonic":2349979.764865633,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.498","gpu_util_pct":2.0,"memory_used_mib":71131.0,"power_w":81.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.510","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.521","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.532","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.542","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.552","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.561","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.570","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.400049,"monotonic":2349980.765684629,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.498","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.509","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.520","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.61,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.530","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.540","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.549","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.559","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.83,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.568","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.4008763,"monotonic":2349981.766512013,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.501","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.513","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.524","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.534","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.543","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.553","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.562","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.99,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.571","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.4016974,"monotonic":2349982.767333105,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.542","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.02,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.574","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.584","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.594","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.603","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.49,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.613","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.4024026,"monotonic":2349983.768038301,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.546","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.557","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.568","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.4,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.579","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.589","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.599","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.608","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.618","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.4031777,"monotonic":2349984.768813559,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.545","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.556","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.567","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.577","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.588","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.597","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.606","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.615","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.4039629,"monotonic":2349985.76959846,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.549","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.560","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.571","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.581","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.591","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.601","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.610","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.619","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.4042404,"monotonic":2349986.769875929,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.545","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.556","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.567","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.577","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.596","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.606","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.615","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.4050221,"monotonic":2349987.770657844,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.575","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.71,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.595","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.605","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.614","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.623","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.405794,"monotonic":2349988.771429615,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.552","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.575","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.584","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.594","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:52.603","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:52.613","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:52.621","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.406392,"monotonic":2349989.772027688,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.552","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.562","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.571","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.579","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.594","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.602","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.609","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.40719,"monotonic":2349990.772825866,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.487","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.499","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.509","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.519","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.529","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.539","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.548","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:54.557","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.4080331,"monotonic":2349991.77366874,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.549","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.560","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.570","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.580","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.590","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.599","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.608","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.4088154,"monotonic":2349992.77445116,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.550","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.561","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.572","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.582","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.592","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.602","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.611","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.621","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.4095967,"monotonic":2349993.775232485,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.557","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.568","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.579","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.589","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.599","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.609","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.618","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.627","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.4103959,"monotonic":2349994.776031561,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.575","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.595","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.604","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.613","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.622","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.411177,"monotonic":2349995.776812691,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.575","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.595","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.604","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.613","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.622","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.4119587,"monotonic":2349996.777594521,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.533","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.544","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.555","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.565","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.575","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.584","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.594","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.603","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.4127626,"monotonic":2349997.778398094,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.552","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.574","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.595","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.604","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.614","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.623","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.4135451,"monotonic":2349998.779181006,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.561","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.572","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.583","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.593","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.603","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.612","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.621","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.630","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.4143205,"monotonic":2349999.77995599,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.556","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.568","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.579","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.589","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.599","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.609","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.618","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.627","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.4151022,"monotonic":2350000.780738116,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.496","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.507","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.518","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.528","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.538","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.547","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.556","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.565","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.4153223,"monotonic":2350001.780958176,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.495","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.506","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.517","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.527","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.537","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.547","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.556","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.565","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.4161663,"monotonic":2350002.781802353,"hcas":{"mlx5_0":{"port_xmit_data":350029912535292,"port_rcv_data":350258420874271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854527776042,"port_rcv_data":1326321867742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.516","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.527","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.538","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.549","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.559","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.568","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.578","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.4169855,"monotonic":2350003.782621377,"hcas":{"mlx5_0":{"port_xmit_data":350029912805317,"port_rcv_data":350258510391656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854528047909,"port_rcv_data":1326411387046,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.535","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":245.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.546","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":216.94,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.558","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":232.78,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.568","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":223.68,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.578","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.596","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.9,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:07.605","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.4177895,"monotonic":2350004.783425165,"hcas":{"mlx5_0":{"port_xmit_data":350030027915195,"port_rcv_data":350258599979890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854643149034,"port_rcv_data":1326500968077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997040093238,"port_rcv_data":350348847067543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956632441177,"port_rcv_data":1255405306056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.518","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":293.33,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.531","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":285.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.543","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":268.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.554","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":266.95,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.565","gpu_util_pct":2.0,"memory_used_mib":71769.0,"power_w":213.2,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.575","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":199.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.585","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":208.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.595","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":226.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.418602,"monotonic":2350005.784237644,"hcas":{"mlx5_0":{"port_xmit_data":350030142893070,"port_rcv_data":350258636894673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854758129286,"port_rcv_data":1326537881188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997274299543,"port_rcv_data":350348847513549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1956866654372,"port_rcv_data":1255405758054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.518","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":267.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.530","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":221.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.540","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":223.49,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.551","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":228.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.561","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":269.63,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.572","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":350.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.582","gpu_util_pct":97.0,"memory_used_mib":71769.0,"power_w":277.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.591","gpu_util_pct":97.0,"memory_used_mib":71769.0,"power_w":351.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.419418,"monotonic":2350006.785053578,"hcas":{"mlx5_0":{"port_xmit_data":350030142893070,"port_rcv_data":350258636894673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854758129286,"port_rcv_data":1326537881188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997557538430,"port_rcv_data":350348848061788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957149917707,"port_rcv_data":1255406332030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.541","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.574","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.585","gpu_util_pct":24.0,"memory_used_mib":71769.0,"power_w":194.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.595","gpu_util_pct":24.0,"memory_used_mib":71769.0,"power_w":194.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.604","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":179.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.613","gpu_util_pct":5.0,"memory_used_mib":71769.0,"power_w":168.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.4202156,"monotonic":2350007.785851435,"hcas":{"mlx5_0":{"port_xmit_data":350030142893070,"port_rcv_data":350258636894673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854758129286,"port_rcv_data":1326537881188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997557538430,"port_rcv_data":350348848061788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957149917707,"port_rcv_data":1255406332030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.500","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.511","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.522","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.533","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.543","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.552","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.27,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.561","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.36,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.570","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.4210594,"monotonic":2350008.786695031,"hcas":{"mlx5_0":{"port_xmit_data":350030197923035,"port_rcv_data":350258636961356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854813157806,"port_rcv_data":1326537946257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997612568207,"port_rcv_data":350348848127975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957204946227,"port_rcv_data":1255406397688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.543","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.554","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.565","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.48,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.576","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.586","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.596","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.605","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.614","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.4218583,"monotonic":2350009.787494237,"hcas":{"mlx5_0":{"port_xmit_data":350030197923035,"port_rcv_data":350258636961356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854813157806,"port_rcv_data":1326537946257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997612568207,"port_rcv_data":350348848127975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957204946227,"port_rcv_data":1255406397688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.525","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.537","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.546","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.52,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.554","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.562","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.571","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.580","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.588","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.42239,"monotonic":2350010.788025707,"hcas":{"mlx5_0":{"port_xmit_data":350030197923035,"port_rcv_data":350258636961356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854813157806,"port_rcv_data":1326537946257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997612568207,"port_rcv_data":350348848127975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957204946227,"port_rcv_data":1255406397688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.502","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.513","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.524","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.534","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.544","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.553","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.563","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.55,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.572","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.4232447,"monotonic":2350011.78888044,"hcas":{"mlx5_0":{"port_xmit_data":350030198171832,"port_rcv_data":350258726457729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854813414091,"port_rcv_data":1326627450072,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997612568207,"port_rcv_data":350348848127975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957204946227,"port_rcv_data":1255406397688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.527","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":126.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.539","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":133.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.550","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":131.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.563","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":148.42,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.573","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.583","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.594","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.9,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.603","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.4240563,"monotonic":2350012.789692089,"hcas":{"mlx5_0":{"port_xmit_data":350030313236804,"port_rcv_data":350258813558022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854928476682,"port_rcv_data":1326714526306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997612568207,"port_rcv_data":350348848127975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957204946227,"port_rcv_data":1255406397688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.525","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":273.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.538","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":277.11,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.550","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":292.28,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.561","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":265.92,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.572","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":163.36,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.582","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":173.79,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.592","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":177.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.602","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":204.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.424867,"monotonic":2350013.79050271,"hcas":{"mlx5_0":{"port_xmit_data":350030313236804,"port_rcv_data":350258813558022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854928476682,"port_rcv_data":1326714526306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997846798380,"port_rcv_data":350348848597726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957439163972,"port_rcv_data":1255406854143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.528","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":215.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.540","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":208.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.551","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":210.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.561","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":204.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.571","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":276.6,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.581","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":284.06,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.590","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":272.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.600","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":271.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.425692,"monotonic":2350014.791327775,"hcas":{"mlx5_0":{"port_xmit_data":350030313236804,"port_rcv_data":350258813558022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854928476682,"port_rcv_data":1326714526306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997962791509,"port_rcv_data":350348848866532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957555162564,"port_rcv_data":1255407141208,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.528","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":96.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.540","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.551","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.561","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.571","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":150.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.580","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":131.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.590","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":151.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.599","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":130.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.4263988,"monotonic":2350015.792034366,"hcas":{"mlx5_0":{"port_xmit_data":350030313236804,"port_rcv_data":350258813558022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1854928476682,"port_rcv_data":1326714526306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997962791509,"port_rcv_data":350348848866532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957555162564,"port_rcv_data":1255407141208,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.506","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.517","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.528","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.538","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.548","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.558","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":84.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.567","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":108.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.576","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":105.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.4272475,"monotonic":2350016.792883484,"hcas":{"mlx5_0":{"port_xmit_data":350030451182461,"port_rcv_data":350258813704529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855029817850,"port_rcv_data":1326714634961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349997962793845,"port_rcv_data":350348848867524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957555168288,"port_rcv_data":1255407142572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.507","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.518","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.529","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.32,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.540","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.550","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.560","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.569","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.11,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.578","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.4273286,"monotonic":2350017.792964122,"hcas":{"mlx5_0":{"port_xmit_data":350030451185053,"port_rcv_data":350258813705521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855029821230,"port_rcv_data":1326714636077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998064138293,"port_rcv_data":350348848977914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957693112176,"port_rcv_data":1255407289218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.530","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.542","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.563","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.572","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.582","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.591","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.69,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:21.600","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.4281447,"monotonic":2350018.793780436,"hcas":{"mlx5_0":{"port_xmit_data":350030451185053,"port_rcv_data":350258813705521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855029821230,"port_rcv_data":1326714636077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998064138766,"port_rcv_data":350348848978439,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957693112176,"port_rcv_data":1255407289218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.508","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.519","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.530","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.39,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.541","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.551","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.561","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.570","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.03,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.579","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.06,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.428997,"monotonic":2350019.794632633,"hcas":{"mlx5_0":{"port_xmit_data":350030566503665,"port_rcv_data":350258980002476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855145166454,"port_rcv_data":1326880958357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998064138766,"port_rcv_data":350348848978439,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957693112176,"port_rcv_data":1255407289218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.540","gpu_util_pct":14.0,"memory_used_mib":71131.0,"power_w":295.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.553","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":181.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":187.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.574","gpu_util_pct":41.0,"memory_used_mib":71131.0,"power_w":179.13,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.584","gpu_util_pct":96.0,"memory_used_mib":71769.0,"power_w":112.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:23.595","gpu_util_pct":96.0,"memory_used_mib":71769.0,"power_w":103.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:23.606","gpu_util_pct":94.0,"memory_used_mib":71769.0,"power_w":106.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:23.616","gpu_util_pct":94.0,"memory_used_mib":71769.0,"power_w":115.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.4297948,"monotonic":2350020.795430571,"hcas":{"mlx5_0":{"port_xmit_data":350030681513788,"port_rcv_data":350259029004006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855260176337,"port_rcv_data":1326929954960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998298329763,"port_rcv_data":350348849408040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1957927312729,"port_rcv_data":1255407729084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.554","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":346.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.566","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":348.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.580","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":338.28,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.591","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":330.38,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.602","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":277.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.614","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":264.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.626","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":291.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.636","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":282.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.4303987,"monotonic":2350021.796034503,"hcas":{"mlx5_0":{"port_xmit_data":350030681748179,"port_rcv_data":350259118484764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855260426636,"port_rcv_data":1327019451750,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998532501980,"port_rcv_data":350348849822242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958161501558,"port_rcv_data":1255408160246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.540","gpu_util_pct":85.0,"memory_used_mib":71131.0,"power_w":242.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.554","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":218.82,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.565","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":222.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.577","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":232.89,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:25.588","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":318.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:25.598","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":350.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:25.609","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":317.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:25.619","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":347.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.431187,"monotonic":2350022.796822521,"hcas":{"mlx5_0":{"port_xmit_data":350030796855833,"port_rcv_data":350259206071845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855375532734,"port_rcv_data":1327107006042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998580631517,"port_rcv_data":350348849942592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958209636440,"port_rcv_data":1255408290580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.538","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":258.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.549","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":266.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.560","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":270.99,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.572","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":270.23,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.583","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":245.25,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.594","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":253.8,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.605","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":224.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.615","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":226.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.4319685,"monotonic":2350023.797604583,"hcas":{"mlx5_0":{"port_xmit_data":350030797107199,"port_rcv_data":350259295569779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855375808440,"port_rcv_data":1327196526301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998814816874,"port_rcv_data":350348850367744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958443839909,"port_rcv_data":1255408733813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.561","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":261.08,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.573","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":264.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.584","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":286.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.596","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":259.93,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.608","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":213.98,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.618","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":226.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.627","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":220.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.637","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":231.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.4327416,"monotonic":2350024.79837732,"hcas":{"mlx5_0":{"port_xmit_data":350030856535864,"port_rcv_data":350259376448083,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855546534509,"port_rcv_data":1327277516935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349998931422724,"port_rcv_data":350348850660077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958560416214,"port_rcv_data":1255409025547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.542","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":285.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.554","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":258.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.566","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":294.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.577","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":291.14,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.588","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":310.63,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.599","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":319.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.609","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":311.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.619","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":290.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.4335415,"monotonic":2350025.799177322,"hcas":{"mlx5_0":{"port_xmit_data":350030856536454,"port_rcv_data":350259376448621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855546534509,"port_rcv_data":1327277516935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999221264009,"port_rcv_data":350348851148699,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958738949299,"port_rcv_data":1255409388589,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.540","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":187.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.552","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":204.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.563","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":182.43,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.574","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":215.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.584","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":283.33,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.595","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":258.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.606","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":276.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.616","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":292.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.4343493,"monotonic":2350026.799985122,"hcas":{"mlx5_0":{"port_xmit_data":350030856536719,"port_rcv_data":350259376448834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855546534509,"port_rcv_data":1327277516935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999329015958,"port_rcv_data":350348851401246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958846713642,"port_rcv_data":1255409657796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.538","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.550","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.561","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.28,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.571","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.581","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":153.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.591","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":138.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.601","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":153.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.610","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":161.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.4343903,"monotonic":2350027.800026153,"hcas":{"mlx5_0":{"port_xmit_data":350030856536719,"port_rcv_data":350259376448834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855546534509,"port_rcv_data":1327277516935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999329016496,"port_rcv_data":350348851401836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958846713642,"port_rcv_data":1255409657796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.533","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.545","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.556","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.98,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.566","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.576","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.585","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.595","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.604","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.4352167,"monotonic":2350028.800852628,"hcas":{"mlx5_0":{"port_xmit_data":350030918290136,"port_rcv_data":350259466013326,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855593529664,"port_rcv_data":1327367070557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999329021682,"port_rcv_data":350348851403698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958846718386,"port_rcv_data":1255409659036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.538","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":199.45,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.550","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":211.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.561","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":243.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.571","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":216.45,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.581","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.590","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.600","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.609","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.4360366,"monotonic":2350029.801672345,"hcas":{"mlx5_0":{"port_xmit_data":350031033382306,"port_rcv_data":350259551975011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855708632897,"port_rcv_data":1327453014419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999375767461,"port_rcv_data":350348851459972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1958908225686,"port_rcv_data":1255409731663,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.536","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":271.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.549","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":262.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.561","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":269.38,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.572","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":292.8,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.583","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":252.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.593","gpu_util_pct":46.0,"memory_used_mib":71769.0,"power_w":243.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.602","gpu_util_pct":5.0,"memory_used_mib":71769.0,"power_w":255.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:33.612","gpu_util_pct":21.0,"memory_used_mib":71769.0,"power_w":245.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.436108,"monotonic":2350030.80174375,"hcas":{"mlx5_0":{"port_xmit_data":350031153923566,"port_rcv_data":350259552106056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855829172945,"port_rcv_data":1327453143751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999609924318,"port_rcv_data":350348851852471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1959142383486,"port_rcv_data":1255410124964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.551","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":107.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.563","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":131.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.574","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":115.92,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.584","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":130.67,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.594","gpu_util_pct":90.0,"memory_used_mib":71769.0,"power_w":278.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.604","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":262.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.614","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":275.56,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.623","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":291.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.4369247,"monotonic":2350031.802560368,"hcas":{"mlx5_0":{"port_xmit_data":350031246079390,"port_rcv_data":350259552204107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855904374435,"port_rcv_data":1327453225467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999844902898,"port_rcv_data":350348852238222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1959377332480,"port_rcv_data":1255410514486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.538","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.550","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.560","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.571","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.581","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":94.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.590","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":90.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.600","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":108.01,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.609","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":85.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.4377444,"monotonic":2350032.803380163,"hcas":{"mlx5_0":{"port_xmit_data":350031282683263,"port_rcv_data":350259641723668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1855924024328,"port_rcv_data":1327542727783,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999920101889,"port_rcv_data":350348852322310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1959469484528,"port_rcv_data":1255410612415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.544","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":239.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.556","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":227.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.567","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":230.35,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.578","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":222.62,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.588","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.598","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.607","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.4383912,"monotonic":2350033.804027193,"hcas":{"mlx5_0":{"port_xmit_data":350031397761065,"port_rcv_data":350259730668874,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856039096700,"port_rcv_data":1327631638598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":349999939516295,"port_rcv_data":350348852343900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1959505852822,"port_rcv_data":1255410650064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.545","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":262.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.559","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":209.49,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.570","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":247.17,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.582","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":254.08,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.593","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":228.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.602","gpu_util_pct":58.0,"memory_used_mib":71769.0,"power_w":256.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.611","gpu_util_pct":16.0,"memory_used_mib":71769.0,"power_w":246.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.620","gpu_util_pct":32.0,"memory_used_mib":71769.0,"power_w":244.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.4392142,"monotonic":2350034.804849203,"hcas":{"mlx5_0":{"port_xmit_data":350031397761065,"port_rcv_data":350259730668874,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856039096700,"port_rcv_data":1327631638598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000173723852,"port_rcv_data":350348852790011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1959740055294,"port_rcv_data":1255411091059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.544","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":121.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.556","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":139.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.567","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":115.72,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.578","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":143.43,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.590","gpu_util_pct":96.0,"memory_used_mib":71769.0,"power_w":289.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.600","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":267.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.609","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":281.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.618","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":266.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.4400256,"monotonic":2350035.805660985,"hcas":{"mlx5_0":{"port_xmit_data":350031513085839,"port_rcv_data":350259909058799,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856154426131,"port_rcv_data":1327810015541,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000292142374,"port_rcv_data":350348853031941,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1959858451080,"port_rcv_data":1255411338626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.544","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":218.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.554","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":219.7,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":276.43,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.573","gpu_util_pct":37.0,"memory_used_mib":71131.0,"power_w":224.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.583","gpu_util_pct":98.0,"memory_used_mib":71769.0,"power_w":114.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.593","gpu_util_pct":47.0,"memory_used_mib":71769.0,"power_w":108.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.604","gpu_util_pct":86.0,"memory_used_mib":71769.0,"power_w":137.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:39.614","gpu_util_pct":71.0,"memory_used_mib":71769.0,"power_w":124.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.4408398,"monotonic":2350036.806475416,"hcas":{"mlx5_0":{"port_xmit_data":350031648935017,"port_rcv_data":350259909204185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856254272519,"port_rcv_data":1327810127094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000292146990,"port_rcv_data":350348853033181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1959858455824,"port_rcv_data":1255411339866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.565","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":257.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.578","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":279.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.589","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":248.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.598","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":294.11,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.606","gpu_util_pct":62.0,"memory_used_mib":71769.0,"power_w":234.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":209.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.627","gpu_util_pct":12.0,"memory_used_mib":71769.0,"power_w":232.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.637","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":208.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.4416504,"monotonic":2350037.807286053,"hcas":{"mlx5_0":{"port_xmit_data":350031648937820,"port_rcv_data":350259909204898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856254275327,"port_rcv_data":1327810127838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000592100315,"port_rcv_data":350348853553832,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960189105467,"port_rcv_data":1255411888389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.579","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.590","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.1,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.612","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.622","gpu_util_pct":87.0,"memory_used_mib":71769.0,"power_w":295.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.632","gpu_util_pct":89.0,"memory_used_mib":71769.0,"power_w":292.17,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.641","gpu_util_pct":90.0,"memory_used_mib":71769.0,"power_w":284.35,"sm_clock_mhz":2407.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.650","gpu_util_pct":90.0,"memory_used_mib":71769.0,"power_w":293.03,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.4424503,"monotonic":2350038.808085577,"hcas":{"mlx5_0":{"port_xmit_data":350031648940026,"port_rcv_data":350259909205638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856254277135,"port_rcv_data":1327810128334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000744571689,"port_rcv_data":350348853848444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960346869983,"port_rcv_data":1255412193362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.542","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.554","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.565","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.64,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.575","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.3,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.585","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.595","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:42.605","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":84.52,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:42.614","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.4432764,"monotonic":2350039.808911841,"hcas":{"mlx5_0":{"port_xmit_data":350031648940026,"port_rcv_data":350259909205638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856254277135,"port_rcv_data":1327810128334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000744571689,"port_rcv_data":350348853848444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960346869983,"port_rcv_data":1255412193362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.566","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.577","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.588","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.49,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.598","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.15,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.607","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.626","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.4440901,"monotonic":2350040.809725676,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.566","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.578","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.589","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.58,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.599","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.609","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.619","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.628","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.637","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.4448998,"monotonic":2350041.810535389,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.544","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.556","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.566","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.35,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.577","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.66,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.587","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:45.598","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:45.608","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:45.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.4457235,"monotonic":2350042.811358951,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.570","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.581","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.592","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.45,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.602","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.612","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.622","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.631","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.640","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.4465008,"monotonic":2350043.812136545,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.549","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.560","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.571","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.49,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.582","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.592","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.602","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.611","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.621","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.44732,"monotonic":2350044.812955929,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.527","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.538","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.549","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.44,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.560","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.570","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.580","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.589","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.598","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.44817,"monotonic":2350045.813805409,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.568","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.580","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.590","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.611","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.620","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.630","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.639","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.4489758,"monotonic":2350046.814611612,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.552","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.563","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.573","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.82,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.583","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.593","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.603","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:50.612","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:50.621","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.4497948,"monotonic":2350047.815430512,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.555","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.567","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.577","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.03,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:51.588","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:51.597","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:51.607","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.616","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.625","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.4504008,"monotonic":2350048.816036583,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.530","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.541","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.552","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.14,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.562","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.571","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.581","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.590","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.86,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.599","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.4512537,"monotonic":2350049.816889513,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.608","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.619","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.59,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.629","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.639","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:53.649","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:53.658","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.58,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.667","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.4520469,"monotonic":2350050.817682491,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.532","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.543","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.554","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.64,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.573","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.583","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.592","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.600","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.452911,"monotonic":2350051.81854639,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.612","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.623","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.58,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.633","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.643","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.653","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.662","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.52,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.671","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.4537013,"monotonic":2350052.819336895,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.590","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.612","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.73,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:56.622","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:56.632","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:56.641","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:56.650","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:56.659","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.4544907,"monotonic":2350053.820126464,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.602","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.613","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.624","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.48,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.634","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.644","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.653","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.662","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.671","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.4552639,"monotonic":2350054.820899508,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.570","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.582","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.593","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.19,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.603","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.613","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:58.623","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.632","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.41,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.641","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.456074,"monotonic":2350055.821709728,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.593","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.604","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.615","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.39,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.625","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.644","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.653","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.39,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.662","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.78,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.4568634,"monotonic":2350056.822499399,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.608","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.619","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:00.629","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:00.639","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:00.649","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:00.658","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:00.667","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.4576447,"monotonic":2350057.82328067,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.600","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.612","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.623","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.633","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.642","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.652","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.661","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.92,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.670","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.4584234,"monotonic":2350058.824059128,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.600","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.612","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.622","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.632","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:02.642","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:02.650","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.658","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.665","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.54,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.459212,"monotonic":2350059.824847805,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.538","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.549","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.560","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.570","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.580","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.589","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.598","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.66,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.607","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.4600596,"monotonic":2350060.825695297,"hcas":{"mlx5_0":{"port_xmit_data":350031893620367,"port_rcv_data":350259909468598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856498956031,"port_rcv_data":1327810391400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.539","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.550","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.561","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.571","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.581","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.591","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.600","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.609","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.4609032,"monotonic":2350061.826538778,"hcas":{"mlx5_0":{"port_xmit_data":350031893888056,"port_rcv_data":350259998983723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856499230443,"port_rcv_data":1327899913513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.566","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":109.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.578","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":110.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.590","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":120.6,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.602","gpu_util_pct":96.0,"memory_used_mib":71131.0,"power_w":104.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.612","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.622","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.631","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.640","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.4617193,"monotonic":2350062.827354803,"hcas":{"mlx5_0":{"port_xmit_data":350032008959695,"port_rcv_data":350260086976956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856614304750,"port_rcv_data":1327987898192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350000989252315,"port_rcv_data":350348854112467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960591548879,"port_rcv_data":1255412453463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.569","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":275.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.581","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":265.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.592","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":302.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.603","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":276.46,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:06.613","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":126.76,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:06.623","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":142.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.633","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":121.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.643","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":136.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.4625247,"monotonic":2350063.828160377,"hcas":{"mlx5_0":{"port_xmit_data":350032008959695,"port_rcv_data":350260086976956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856614304750,"port_rcv_data":1327987898192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001223475298,"port_rcv_data":350348854574113,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960825769438,"port_rcv_data":1255412912576,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.568","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":212.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.580","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":205.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.591","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":201.3,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":223.71,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.611","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":265.99,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.622","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":283.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.632","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":260.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.643","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":295.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.46333,"monotonic":2350064.828965787,"hcas":{"mlx5_0":{"port_xmit_data":350032008959695,"port_rcv_data":350260086976956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856614304750,"port_rcv_data":1327987898192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001340642028,"port_rcv_data":350348854818686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960942921414,"port_rcv_data":1255413163864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.565","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.577","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.588","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":85.67,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.598","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":99.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.608","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":186.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.618","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":170.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.627","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":213.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.636","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":182.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.464153,"monotonic":2350065.829788617,"hcas":{"mlx5_0":{"port_xmit_data":350032008959695,"port_rcv_data":350260086976956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856614304750,"port_rcv_data":1327987898192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001340642028,"port_rcv_data":350348854818686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960942921414,"port_rcv_data":1255413163864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.575","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.586","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.597","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.607","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.626","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":111.99,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":102.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259210.4649866,"monotonic":2350066.830622143,"hcas":{"mlx5_0":{"port_xmit_data":350032008959695,"port_rcv_data":350260086976956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856614304750,"port_rcv_data":1327987898192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001340642028,"port_rcv_data":350348854818686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960942921414,"port_rcv_data":1255413163864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.568","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.580","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.590","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.23,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.610","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.620","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.629","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.638","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.65,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.465819,"monotonic":2350067.83145456,"hcas":{"mlx5_0":{"port_xmit_data":350032129500374,"port_rcv_data":350260087107720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856734844606,"port_rcv_data":1327988026966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001340646708,"port_rcv_data":350348854819926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1960942926094,"port_rcv_data":1255413165104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.569","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.580","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.591","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.29,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.602","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.611","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:11.621","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.630","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.85,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.639","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.4663944,"monotonic":2350068.832030068,"hcas":{"mlx5_0":{"port_xmit_data":350032129505054,"port_rcv_data":350260087108960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856734849286,"port_rcv_data":1327988028206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001461186712,"port_rcv_data":350348854951938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961063465950,"port_rcv_data":1255413295350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.568","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.580","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.590","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.31,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.611","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.621","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.630","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.73,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.640","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.467215,"monotonic":2350069.832850881,"hcas":{"mlx5_0":{"port_xmit_data":350032129755408,"port_rcv_data":350260176606502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856735107927,"port_rcv_data":1328077534346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001461187152,"port_rcv_data":350348854952430,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961063465950,"port_rcv_data":1255413295350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.547","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":231.35,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.559","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":236.34,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.571","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":256.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.581","gpu_util_pct":22.0,"memory_used_mib":71131.0,"power_w":235.83,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.591","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:13.601","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.612","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.34,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.623","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.4680474,"monotonic":2350070.833683363,"hcas":{"mlx5_0":{"port_xmit_data":350032244817655,"port_rcv_data":350260261787920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856850169224,"port_rcv_data":1328162670890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001695381189,"port_rcv_data":350348855384975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961297672142,"port_rcv_data":1255413740484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.564","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":273.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.575","gpu_util_pct":46.0,"memory_used_mib":71131.0,"power_w":288.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.585","gpu_util_pct":41.0,"memory_used_mib":71131.0,"power_w":278.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.595","gpu_util_pct":92.0,"memory_used_mib":71131.0,"power_w":296.75,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.605","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":268.33,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.615","gpu_util_pct":69.0,"memory_used_mib":71769.0,"power_w":283.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.625","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":250.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.636","gpu_util_pct":95.0,"memory_used_mib":71769.0,"power_w":313.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.4688747,"monotonic":2350071.834510453,"hcas":{"mlx5_0":{"port_xmit_data":350032244817655,"port_rcv_data":350260261787920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856850169224,"port_rcv_data":1328162670890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001808818042,"port_rcv_data":350348855652882,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961411064960,"port_rcv_data":1255414004896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.572","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.584","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":100.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:15.594","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":99.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:15.608","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:15.618","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":269.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.628","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":237.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.637","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":254.92,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.647","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":253.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.4696925,"monotonic":2350072.835328192,"hcas":{"mlx5_0":{"port_xmit_data":350032244817655,"port_rcv_data":350260261787920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856850169224,"port_rcv_data":1328162670890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001808818042,"port_rcv_data":350348855652882,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961411064960,"port_rcv_data":1255414004896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.573","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.64,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.606","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.616","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":87.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.625","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":86.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.634","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":105.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.643","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":103.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.4705172,"monotonic":2350073.836153024,"hcas":{"mlx5_0":{"port_xmit_data":350032244817655,"port_rcv_data":350260261787920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856850169224,"port_rcv_data":1328162670890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001808818042,"port_rcv_data":350348855652882,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961411064960,"port_rcv_data":1255414004896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.550","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.561","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.572","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.582","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.592","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.601","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.610","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.54,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.619","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.4713628,"monotonic":2350074.836998576,"hcas":{"mlx5_0":{"port_xmit_data":350032244817655,"port_rcv_data":350260261787920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856850169224,"port_rcv_data":1328162670890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001808818042,"port_rcv_data":350348855652882,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961411064960,"port_rcv_data":1255414004896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.573","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.584","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.595","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.615","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.625","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.634","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.9,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.643","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.4721885,"monotonic":2350075.837824366,"hcas":{"mlx5_0":{"port_xmit_data":350032245072075,"port_rcv_data":350260351289653,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1856850437310,"port_rcv_data":1328252186102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350001808818042,"port_rcv_data":350348855652882,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961411064960,"port_rcv_data":1255414004896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.553","gpu_util_pct":95.0,"memory_used_mib":71131.0,"power_w":211.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.566","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":211.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.578","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":239.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.587","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":222.69,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.596","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.603","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.611","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.75,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.618","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.4729261,"monotonic":2350076.838561647,"hcas":{"mlx5_0":{"port_xmit_data":350032495226574,"port_rcv_data":350260529087836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857064889510,"port_rcv_data":1328429917482,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002030252491,"port_rcv_data":350348856074929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961636834291,"port_rcv_data":1255414464630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.584","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":325.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.596","gpu_util_pct":89.0,"memory_used_mib":71131.0,"power_w":319.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.608","gpu_util_pct":88.0,"memory_used_mib":71131.0,"power_w":319.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.621","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":321.99,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:20.633","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":227.42,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.643","gpu_util_pct":22.0,"memory_used_mib":71769.0,"power_w":229.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.652","gpu_util_pct":63.0,"memory_used_mib":71769.0,"power_w":282.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.662","gpu_util_pct":48.0,"memory_used_mib":71769.0,"power_w":278.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.4737225,"monotonic":2350077.839357677,"hcas":{"mlx5_0":{"port_xmit_data":350032610296026,"port_rcv_data":350260617534008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857179959370,"port_rcv_data":1328518354908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002259549476,"port_rcv_data":350348856539626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1961897480431,"port_rcv_data":1255414937317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.596","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":275.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.610","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":268.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.621","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":279.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.633","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":259.33,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.645","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":330.9,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.655","gpu_util_pct":98.0,"memory_used_mib":71769.0,"power_w":325.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.665","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":333.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.674","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":311.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.4744785,"monotonic":2350078.84011435,"hcas":{"mlx5_0":{"port_xmit_data":350032610296026,"port_rcv_data":350260617534008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857179959370,"port_rcv_data":1328518354908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002493734026,"port_rcv_data":350348856963449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962131669333,"port_rcv_data":1255415365880,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.594","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":235.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.606","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":235.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.617","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":248.24,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.628","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":213.11,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:22.639","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":258.24,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.649","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":292.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.659","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":266.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.669","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":279.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.4752474,"monotonic":2350079.84088313,"hcas":{"mlx5_0":{"port_xmit_data":350032610296026,"port_rcv_data":350260617534008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857179959370,"port_rcv_data":1328518354908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002611562433,"port_rcv_data":350348857263498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962249479513,"port_rcv_data":1255415660183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.555","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":107.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.567","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":103.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.578","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":106.79,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.589","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.04,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.599","gpu_util_pct":28.0,"memory_used_mib":71769.0,"power_w":211.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.609","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":234.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.619","gpu_util_pct":78.0,"memory_used_mib":71769.0,"power_w":191.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.628","gpu_util_pct":93.0,"memory_used_mib":71769.0,"power_w":217.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.4760566,"monotonic":2350080.841692497,"hcas":{"mlx5_0":{"port_xmit_data":350032610296026,"port_rcv_data":350260617534008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857179959370,"port_rcv_data":1328518354908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002611562433,"port_rcv_data":350348857263498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962249479513,"port_rcv_data":1255415660183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.573","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.0,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.607","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.626","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.636","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.645","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.4768653,"monotonic":2350081.842501066,"hcas":{"mlx5_0":{"port_xmit_data":350032610296291,"port_rcv_data":350260617534221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857179959370,"port_rcv_data":1328518354908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002611562433,"port_rcv_data":350348857263498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962249479513,"port_rcv_data":1255415660183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.595","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.607","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.618","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.04,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:25.629","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:25.640","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:25.650","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.660","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.11,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.669","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.4776661,"monotonic":2350082.843301554,"hcas":{"mlx5_0":{"port_xmit_data":350032870884155,"port_rcv_data":350260617812532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857403346430,"port_rcv_data":1328518594083,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002834951125,"port_rcv_data":350348857505857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962510065509,"port_rcv_data":1255415935943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.582","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.594","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.85,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.615","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.625","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.645","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.654","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.4784858,"monotonic":2350083.844121146,"hcas":{"mlx5_0":{"port_xmit_data":350032870884155,"port_rcv_data":350260617812532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857403346430,"port_rcv_data":1328518594083,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002834951125,"port_rcv_data":350348857505857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962510065509,"port_rcv_data":1255415935943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.580","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:27.592","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:27.603","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.98,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:27.613","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.623","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.633","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.642","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.651","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.4793057,"monotonic":2350084.844941514,"hcas":{"mlx5_0":{"port_xmit_data":350032871147954,"port_rcv_data":350260707323659,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857403615844,"port_rcv_data":1328608110686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002834951125,"port_rcv_data":350348857505857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962510065509,"port_rcv_data":1255415935943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.583","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":98.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.595","gpu_util_pct":92.0,"memory_used_mib":71131.0,"power_w":97.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.607","gpu_util_pct":95.0,"memory_used_mib":71131.0,"power_w":126.4,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.619","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":92.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.630","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.639","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.648","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.657","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.4801311,"monotonic":2350085.845766727,"hcas":{"mlx5_0":{"port_xmit_data":350032986243798,"port_rcv_data":350260793461056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857518716999,"port_rcv_data":1328694241347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350002834951125,"port_rcv_data":350348857505857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962510065509,"port_rcv_data":1255415935943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.568","gpu_util_pct":99.0,"memory_used_mib":71131.0,"power_w":282.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.581","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":276.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.593","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":303.14,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.604","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":299.85,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.616","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":117.87,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.628","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":130.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.638","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":116.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.649","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":137.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.480951,"monotonic":2350086.846586861,"hcas":{"mlx5_0":{"port_xmit_data":350032986243798,"port_rcv_data":350260793461056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857518716999,"port_rcv_data":1328694241347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003069169845,"port_rcv_data":350348857962930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962744272756,"port_rcv_data":1255416381621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.582","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":221.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.594","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":211.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":230.94,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.616","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":224.9,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.626","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":284.97,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.638","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":271.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.648","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":295.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.658","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":304.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.481763,"monotonic":2350087.84739853,"hcas":{"mlx5_0":{"port_xmit_data":350032986243798,"port_rcv_data":350260793461056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857518716999,"port_rcv_data":1328694241347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003183864401,"port_rcv_data":350348858248414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962858947538,"port_rcv_data":1255416662640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.598","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.609","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.620","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.17,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.631","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.641","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":193.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.651","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":194.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.661","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":161.09,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.670","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":164.33,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.4825683,"monotonic":2350088.848204097,"hcas":{"mlx5_0":{"port_xmit_data":350032986243798,"port_rcv_data":350260793461056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857518716999,"port_rcv_data":1328694241347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003183864401,"port_rcv_data":350348858248414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962858947538,"port_rcv_data":1255416662640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.616","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.627","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.08,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.638","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.648","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.658","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.667","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.16,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.676","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.05,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.4833825,"monotonic":2350089.849018086,"hcas":{"mlx5_0":{"port_xmit_data":350033104989629,"port_rcv_data":350260793588645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857637461943,"port_rcv_data":1328694369671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003183869081,"port_rcv_data":350348858249654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962858952218,"port_rcv_data":1255416663880,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.583","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.594","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.604","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.5,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:33.615","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.625","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.644","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.653","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.4842174,"monotonic":2350090.849853013,"hcas":{"mlx5_0":{"port_xmit_data":350033104994309,"port_rcv_data":350260793589885,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857637466623,"port_rcv_data":1328694370911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003302614744,"port_rcv_data":350348858378026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962977697162,"port_rcv_data":1255416789972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.597","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.608","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.619","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.629","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.638","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.648","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.92,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.657","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.485047,"monotonic":2350091.850682568,"hcas":{"mlx5_0":{"port_xmit_data":350033104994309,"port_rcv_data":350260793589885,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857637466623,"port_rcv_data":1328694370911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003302614744,"port_rcv_data":350348858378026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962977697162,"port_rcv_data":1255416789972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.564","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.576","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:35.587","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.04,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:35.597","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.607","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.617","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.626","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.4858947,"monotonic":2350092.851530227,"hcas":{"mlx5_0":{"port_xmit_data":350033105237311,"port_rcv_data":350260883080602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857637726173,"port_rcv_data":1328783877278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003302614744,"port_rcv_data":350348858378026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962977697162,"port_rcv_data":1255416789972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.592","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":138.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.604","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":135.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.616","gpu_util_pct":97.0,"memory_used_mib":71131.0,"power_w":134.85,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.627","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":150.68,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.638","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.647","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.656","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.665","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.4867108,"monotonic":2350093.852346262,"hcas":{"mlx5_0":{"port_xmit_data":350033220318783,"port_rcv_data":350260971784544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857752803745,"port_rcv_data":1328872566342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003302614744,"port_rcv_data":350348858378026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1962977697162,"port_rcv_data":1255416789972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.613","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":280.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.625","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":287.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:37.636","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":279.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:37.646","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":266.03,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.656","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":160.88,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.667","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":160.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.677","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":125.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.687","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":199.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.4874969,"monotonic":2350094.853132834,"hcas":{"mlx5_0":{"port_xmit_data":350033220318783,"port_rcv_data":350260971784544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857752803745,"port_rcv_data":1328872566342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003536826801,"port_rcv_data":350348858829026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963211901404,"port_rcv_data":1255417232366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.593","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":208.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":179.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.616","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":205.42,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.627","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":191.95,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.637","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":272.43,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.647","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":281.03,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.659","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":261.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.670","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":291.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.488302,"monotonic":2350095.853937907,"hcas":{"mlx5_0":{"port_xmit_data":350033220318783,"port_rcv_data":350260971784544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857752803745,"port_rcv_data":1328872566342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003654982533,"port_rcv_data":350348859131538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963329998662,"port_rcv_data":1255417496062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.588","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":99.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.600","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.610","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.57,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.621","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.75,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.630","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":199.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.640","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":196.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.650","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":181.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.659","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":179.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.489123,"monotonic":2350096.854758993,"hcas":{"mlx5_0":{"port_xmit_data":350033220318783,"port_rcv_data":350260971784544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857752803745,"port_rcv_data":1328872566342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003654982533,"port_rcv_data":350348859131538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963329998662,"port_rcv_data":1255417496062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.589","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.601","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.612","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.49,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:40.622","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:40.632","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":99.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.642","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":91.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.651","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.661","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":85.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.4899454,"monotonic":2350097.855581092,"hcas":{"mlx5_0":{"port_xmit_data":350033220318783,"port_rcv_data":350260971784544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857752803745,"port_rcv_data":1328872566342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003654982533,"port_rcv_data":350348859131538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963329998662,"port_rcv_data":1255417496062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.593","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.616","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.626","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.636","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.645","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.654","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.663","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.4907663,"monotonic":2350098.856402293,"hcas":{"mlx5_0":{"port_xmit_data":350033220318783,"port_rcv_data":350260971784544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857752803745,"port_rcv_data":1328872566342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003654982533,"port_rcv_data":350348859131538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963329998662,"port_rcv_data":1255417496062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.634","gpu_util_pct":70.0,"memory_used_mib":71131.0,"power_w":81.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.646","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.657","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":90.78,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.666","gpu_util_pct":7.0,"memory_used_mib":71131.0,"power_w":92.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.675","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.683","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.691","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.700","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.4915478,"monotonic":2350099.857183614,"hcas":{"mlx5_0":{"port_xmit_data":350033335681958,"port_rcv_data":350261148016268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857868166082,"port_rcv_data":1329048777825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003654982533,"port_rcv_data":350348859131538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963329998662,"port_rcv_data":1255417496062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.595","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":304.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.609","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":274.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.622","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":285.92,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.633","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":294.01,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.646","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":107.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.656","gpu_util_pct":88.0,"memory_used_mib":71769.0,"power_w":118.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.666","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":116.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.676","gpu_util_pct":98.0,"memory_used_mib":71769.0,"power_w":116.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.492348,"monotonic":2350100.857983638,"hcas":{"mlx5_0":{"port_xmit_data":350033475719264,"port_rcv_data":350261148167342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857971003990,"port_rcv_data":1329048890355,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350003889185621,"port_rcv_data":350348859570146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963564209678,"port_rcv_data":1255417941757,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":245.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.608","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":243.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.619","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":251.71,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.630","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":224.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.640","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":285.15,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.651","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":265.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.661","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":292.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.670","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":286.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.493171,"monotonic":2350101.858806698,"hcas":{"mlx5_0":{"port_xmit_data":350033475724008,"port_rcv_data":350261148168582,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857971008606,"port_rcv_data":1329048891595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004107476719,"port_rcv_data":350348859960943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963819683237,"port_rcv_data":1255418370269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.600","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.611","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":108.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.622","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":114.51,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.633","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":84.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.643","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":227.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.653","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":217.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.662","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":204.01,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.672","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":204.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.4939861,"monotonic":2350102.859621861,"hcas":{"mlx5_0":{"port_xmit_data":350033475724008,"port_rcv_data":350261148168582,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857971008606,"port_rcv_data":1329048891595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004107477257,"port_rcv_data":350348859961533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963819683237,"port_rcv_data":1255418370269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.593","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:46.615","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.99,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.626","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.636","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.646","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.656","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.665","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":94.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.4948099,"monotonic":2350103.860445505,"hcas":{"mlx5_0":{"port_xmit_data":350033475724008,"port_rcv_data":350261148168582,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857971008606,"port_rcv_data":1329048891595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004107477257,"port_rcv_data":350348859961533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963819683237,"port_rcv_data":1255418370269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.574","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.606","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.616","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.626","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.644","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.4956563,"monotonic":2350104.861291969,"hcas":{"mlx5_0":{"port_xmit_data":350033475724008,"port_rcv_data":350261148168582,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857971008606,"port_rcv_data":1329048891595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004107477257,"port_rcv_data":350348859961533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963819683237,"port_rcv_data":1255418370269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.607","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.618","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.0,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.628","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.638","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.648","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.657","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.666","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.4964776,"monotonic":2350105.862113271,"hcas":{"mlx5_0":{"port_xmit_data":350033475981741,"port_rcv_data":350261237673209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1857971276112,"port_rcv_data":1329138406460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004107477257,"port_rcv_data":350348859961533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1963819683237,"port_rcv_data":1255418370269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.634","gpu_util_pct":35.0,"memory_used_mib":71131.0,"power_w":191.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.647","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":192.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.659","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":227.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.671","gpu_util_pct":12.0,"memory_used_mib":71131.0,"power_w":195.57,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.682","gpu_util_pct":87.0,"memory_used_mib":71769.0,"power_w":95.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.694","gpu_util_pct":98.0,"memory_used_mib":71769.0,"power_w":98.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.705","gpu_util_pct":95.0,"memory_used_mib":71769.0,"power_w":102.65,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.714","gpu_util_pct":98.0,"memory_used_mib":71769.0,"power_w":101.17,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.4972544,"monotonic":2350106.86289019,"hcas":{"mlx5_0":{"port_xmit_data":350033709812674,"port_rcv_data":350261327004486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858205131334,"port_rcv_data":1329227736807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004341686552,"port_rcv_data":350348860407446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964053894839,"port_rcv_data":1255418817528,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.634","gpu_util_pct":6.0,"memory_used_mib":71131.0,"power_w":309.56,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.645","gpu_util_pct":71.0,"memory_used_mib":71131.0,"power_w":289.75,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.656","gpu_util_pct":66.0,"memory_used_mib":71131.0,"power_w":307.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.667","gpu_util_pct":30.0,"memory_used_mib":71131.0,"power_w":290.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.678","gpu_util_pct":99.0,"memory_used_mib":71769.0,"power_w":270.08,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.691","gpu_util_pct":42.0,"memory_used_mib":71769.0,"power_w":272.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.702","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":275.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.712","gpu_util_pct":68.0,"memory_used_mib":71769.0,"power_w":256.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.498031,"monotonic":2350107.863666492,"hcas":{"mlx5_0":{"port_xmit_data":350033709817354,"port_rcv_data":350261327005726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858205136014,"port_rcv_data":1329227738047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004579259365,"port_rcv_data":350348860853898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964291410409,"port_rcv_data":1255419232827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.615","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.627","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.638","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":107.36,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:51.649","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.48,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.659","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":236.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.668","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":279.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.678","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":262.98,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.688","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":284.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.4988313,"monotonic":2350108.864467134,"hcas":{"mlx5_0":{"port_xmit_data":350033709817354,"port_rcv_data":350261327005726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858205136014,"port_rcv_data":1329227738047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004579259838,"port_rcv_data":350348860854423,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964291410409,"port_rcv_data":1255419232827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:52.603","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:52.614","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:52.625","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.92,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.635","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.645","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.655","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.664","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.673","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":108.28,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.4996593,"monotonic":2350109.86529505,"hcas":{"mlx5_0":{"port_xmit_data":350033832152879,"port_rcv_data":350261327135975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858327470782,"port_rcv_data":1329227866418,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004579264629,"port_rcv_data":350348860855813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964291415089,"port_rcv_data":1255419234067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.604","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.614","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.623","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.89,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.632","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.641","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.650","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.659","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.667","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.93,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.5004847,"monotonic":2350110.866120295,"hcas":{"mlx5_0":{"port_xmit_data":350033832157559,"port_rcv_data":350261327137215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858327475462,"port_rcv_data":1329227867658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004701600069,"port_rcv_data":350348860986053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964413749857,"port_rcv_data":1255419363415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.583","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.22,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.594","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.616","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.625","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.635","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.644","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.653","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.501327,"monotonic":2350111.86696301,"hcas":{"mlx5_0":{"port_xmit_data":350033832157559,"port_rcv_data":350261327137215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858327475462,"port_rcv_data":1329227867658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004701600069,"port_rcv_data":350348860986053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964413749857,"port_rcv_data":1255419363415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.605","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.617","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.627","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.0,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.637","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.647","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.656","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.665","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.674","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.5021462,"monotonic":2350112.867782133,"hcas":{"mlx5_0":{"port_xmit_data":350033832157559,"port_rcv_data":350261327137215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858327475462,"port_rcv_data":1329227867658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004701600069,"port_rcv_data":350348860986053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964413749857,"port_rcv_data":1255419363415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.581","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.592","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:56.603","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.613","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.623","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.633","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.642","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.651","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.5029912,"monotonic":2350113.868626924,"hcas":{"mlx5_0":{"port_xmit_data":350033832157559,"port_rcv_data":350261327137215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858327475462,"port_rcv_data":1329227867658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004701600069,"port_rcv_data":350348860986053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964413749857,"port_rcv_data":1255419363415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.644","gpu_util_pct":81.0,"memory_used_mib":71131.0,"power_w":80.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.656","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.668","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.96,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.680","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":97.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.690","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.700","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.709","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.718","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.503761,"monotonic":2350114.869397459,"hcas":{"mlx5_0":{"port_xmit_data":350033947513028,"port_rcv_data":350261504474360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858442833431,"port_rcv_data":1329405185594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004701600069,"port_rcv_data":350348860986053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964413749857,"port_rcv_data":1255419363415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.606","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":290.27,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.618","gpu_util_pct":98.0,"memory_used_mib":71131.0,"power_w":271.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.630","gpu_util_pct":99.0,"memory_used_mib":71131.0,"power_w":281.78,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.642","gpu_util_pct":100.0,"memory_used_mib":71131.0,"power_w":262.79,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.653","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":122.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.663","gpu_util_pct":94.0,"memory_used_mib":71769.0,"power_w":114.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.673","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":133.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.684","gpu_util_pct":96.0,"memory_used_mib":71769.0,"power_w":114.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.5045717,"monotonic":2350115.870207281,"hcas":{"mlx5_0":{"port_xmit_data":350033947513028,"port_rcv_data":350261504474360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858442833431,"port_rcv_data":1329405185594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350004935788564,"port_rcv_data":350348861413025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964647949435,"port_rcv_data":1255419801841,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.608","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":236.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.619","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":218.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.630","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":247.77,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.640","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":238.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.650","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":270.93,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.660","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":283.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.670","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":260.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.679","gpu_util_pct":100.0,"memory_used_mib":71769.0,"power_w":279.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.50539,"monotonic":2350116.871025432,"hcas":{"mlx5_0":{"port_xmit_data":350033947513028,"port_rcv_data":350261504474360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858442833431,"port_rcv_data":1329405185594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005052701946,"port_rcv_data":350348861670281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964764853016,"port_rcv_data":1255420061844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:00.608","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":88.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:00.619","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.630","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":86.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.640","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":97.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.650","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":220.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.659","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":231.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.668","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":226.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.677","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":231.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.5062113,"monotonic":2350117.871847148,"hcas":{"mlx5_0":{"port_xmit_data":350033947513028,"port_rcv_data":350261504474360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858442833431,"port_rcv_data":1329405185594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005052701946,"port_rcv_data":350348861670281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964764853016,"port_rcv_data":1255420061844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.585","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.596","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.607","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.98,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.617","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.626","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.636","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":83.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.645","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":107.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.653","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":82.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.5064144,"monotonic":2350118.87204988,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.610","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.621","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.632","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.643","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.652","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.662","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.671","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.680","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.5072362,"monotonic":2350119.872872191,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.646","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.657","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.668","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.81,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.679","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.689","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.699","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.708","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.718","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.508031,"monotonic":2350120.873666537,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.625","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.637","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.647","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.95,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.657","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.667","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.676","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.685","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.694","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.5088377,"monotonic":2350121.874473204,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.648","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.659","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:05.670","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.54,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.680","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.690","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.699","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.708","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.717","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.5096183,"monotonic":2350122.875253971,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.633","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.644","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.655","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":83.55,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.665","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.674","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.683","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.692","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.701","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.5104167,"monotonic":2350123.876052437,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.652","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.664","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.674","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.55,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.684","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.694","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.703","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.712","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.99,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.720","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.5111957,"monotonic":2350124.876831734,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:08.629","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:08.640","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.650","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.660","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.670","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.679","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.688","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.697","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.512023,"monotonic":2350125.877658529,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.653","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.665","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.676","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.686","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.696","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.706","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.716","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.725","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.512808,"monotonic":2350126.87844406,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.630","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.642","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.653","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.663","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.673","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.683","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.692","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.701","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.5136166,"monotonic":2350127.879252359,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.657","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:11.668","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.679","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.47,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.689","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.699","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.709","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.718","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.727","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.5143971,"monotonic":2350128.880033041,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:12.637","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.648","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.659","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.669","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.679","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.689","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.698","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.707","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.5151987,"monotonic":2350129.880834418,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.658","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.669","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.680","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.690","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.700","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.710","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.719","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.71,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.728","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.5159788,"monotonic":2350130.881614597,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.661","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.672","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.683","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.694","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.704","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.714","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.723","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.733","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.5167546,"monotonic":2350131.882390428,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.662","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.673","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.684","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.695","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.705","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.714","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.723","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.732","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.5175278,"monotonic":2350132.88316378,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:16.665","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:16.676","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:16.687","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.697","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.706","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.716","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.725","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.734","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.518301,"monotonic":2350133.883936792,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.661","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.673","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.683","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.51,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.694","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.703","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.713","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.722","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.731","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.5190754,"monotonic":2350134.884711198,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:18.641","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:18.652","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.662","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.76,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.672","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.682","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.691","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.700","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.709","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.5198758,"monotonic":2350135.885511798,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.633","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.644","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.654","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.41,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.664","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.674","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.683","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.692","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.701","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.5206847,"monotonic":2350136.886320476,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.638","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.649","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.659","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.669","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.679","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.688","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.697","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.44,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.706","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.5214915,"monotonic":2350137.887127149,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.648","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.660","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.671","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.681","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.691","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.700","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.710","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.42,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.719","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.5223024,"monotonic":2350138.887937774,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:22.662","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:22.674","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.685","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.695","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.705","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.715","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.724","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.733","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.5230834,"monotonic":2350139.888719306,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.670","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.681","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.692","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.702","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.712","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.721","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.730","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.739","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.5238595,"monotonic":2350140.889495385,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.637","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.649","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.659","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.36,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.670","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.680","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.690","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.699","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.708","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.5246713,"monotonic":2350141.890307013,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.673","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.684","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.695","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.39,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.705","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.715","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.724","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.733","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.742","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.5254486,"monotonic":2350142.891084024,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.665","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.676","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.687","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.41,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.697","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.707","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.716","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.725","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.734","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.5262325,"monotonic":2350143.891868166,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:27.666","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.678","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.689","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.699","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.709","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.718","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.728","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.737","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.5270135,"monotonic":2350144.892649389,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.646","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.657","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.668","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.678","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.688","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.698","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.707","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.716","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.5278187,"monotonic":2350145.893454462,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.674","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.685","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.695","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.706","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.716","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.725","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.734","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.743","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.5285978,"monotonic":2350146.89423357,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:30.670","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:30.681","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.692","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.702","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.712","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.722","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.731","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.740","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.5293748,"monotonic":2350147.895010472,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.653","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.665","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.675","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.44,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.685","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.695","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.704","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.713","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.722","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.5301783,"monotonic":2350148.895813822,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.673","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.684","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.695","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.704","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.714","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.723","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.732","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.741","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.530962,"monotonic":2350149.896597771,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.677","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.688","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.699","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.709","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.719","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.729","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.738","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.747","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.5317392,"monotonic":2350150.89737479,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.669","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.680","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.691","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.701","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.711","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.720","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.729","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.738","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.5325265,"monotonic":2350151.898162199,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.672","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.684","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.695","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.705","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.715","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":78.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.725","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.734","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.743","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.533307,"monotonic":2350152.898942791,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.678","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.689","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.700","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.710","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.720","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.729","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.739","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.748","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.534087,"monotonic":2350153.899722584,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.681","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.692","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.703","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.714","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.724","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.733","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.743","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.752","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.5348616,"monotonic":2350154.900497539,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.676","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.688","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.698","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.708","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":76.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.718","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.728","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.737","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.746","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.5356438,"monotonic":2350155.90127952,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.649","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.660","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.671","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.681","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.691","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.701","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.710","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:39.719","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.5364554,"monotonic":2350156.902091254,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.677","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.688","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.698","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":81.93,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.709","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.719","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.728","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:40.736","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.744","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.5372477,"monotonic":2350157.902883655,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.676","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.687","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.698","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.42,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.708","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.717","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.726","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.736","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.745","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259302.5380356,"monotonic":2350158.903671461,"hcas":{"mlx5_0":{"port_xmit_data":350034068058354,"port_rcv_data":350261504603994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1858563377967,"port_rcv_data":1329405315221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":350005173247168,"port_rcv_data":350348861802303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1964885397552,"port_rcv_data":1255420191579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:42.664","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:42.676","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:42.686","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":82.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:42.697","gpu_util_pct":0.0,"memory_used_mib":71131.0,"power_w":77.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:42.706","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":77.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:42.716","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:42.725","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:42.734","gpu_util_pct":0.0,"memory_used_mib":71769.0,"power_w":80.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_4.jsonl

@@ -0,0 +1,459 @@
+{"wall_time":1788258844.2412958,"monotonic":2349692.688309665,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:04.385","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:04.401","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:04.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:04.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:04.433","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:04.443","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:04.452","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:04.461","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258845.2420702,"monotonic":2349693.689083912,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.384","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.417","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.427","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.2428524,"monotonic":2349694.689866295,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.360","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.371","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.382","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.393","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.404","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.413","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.423","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.432","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.243655,"monotonic":2349695.690668622,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.383","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.417","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.427","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.436","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.2444382,"monotonic":2349696.691452124,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.391","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.2450142,"monotonic":2349697.692028013,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.392","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.2457898,"monotonic":2349698.692803578,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.391","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.245996,"monotonic":2349699.693008798,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.384","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.416","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.58,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.425","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.434","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.443","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.452","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.2467902,"monotonic":2349700.693803846,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.389","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.400","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.420","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.430","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.439","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.456","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.2475765,"monotonic":2349701.694590394,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.391","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.402","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.413","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.424","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.433","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.443","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.451","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.460","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.2483647,"monotonic":2349702.695378443,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.394","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.430","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.438","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.2491612,"monotonic":2349703.69617489,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.409","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.419","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.429","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.438","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.2499542,"monotonic":2349704.696967943,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.396","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.417","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.427","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.25074,"monotonic":2349705.697753788,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.396","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.408","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.419","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.429","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.439","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.449","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.458","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.467","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.2515218,"monotonic":2349706.698535426,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.399","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.432","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.468","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.2523034,"monotonic":2349707.699317192,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.417","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.427","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.2530918,"monotonic":2349708.700105424,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.400","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.411","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.422","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.433","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.443","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":82.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.462","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.471","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.2538593,"monotonic":2349709.700873086,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.409","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.420","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.431","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.468","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.2546432,"monotonic":2349710.701656854,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.401","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.434","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.444","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.472","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.255423,"monotonic":2349711.702436743,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.396","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.418","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.429","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.439","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.449","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.458","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.467","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.2562125,"monotonic":2349712.703226078,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.435","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.471","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.2570143,"monotonic":2349713.704028013,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.396","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.418","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.429","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.439","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.448","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.457","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.466","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.257803,"monotonic":2349714.70481686,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.435","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.472","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.2585864,"monotonic":2349715.705600101,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.399","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.431","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.467","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.2593763,"monotonic":2349716.706390221,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.399","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.431","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.440","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.449","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.458","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.466","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.2599862,"monotonic":2349717.706999428,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.426","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.437","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.457","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.466","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.475","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.2607667,"monotonic":2349718.707780366,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.400","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.411","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.422","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.433","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.442","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.452","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.461","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.470","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.2615523,"monotonic":2349719.708566209,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.426","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.437","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.457","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.466","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.475","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.2623348,"monotonic":2349720.709348533,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.426","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.437","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.456","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.465","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.474","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.2631161,"monotonic":2349721.710129906,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.419","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.430","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.440","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.468","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.477","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.2638996,"monotonic":2349722.71091342,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.419","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.430","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.440","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.468","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.477","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.2646816,"monotonic":2349723.711695308,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.411","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.434","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.444","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.464","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.473","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.481","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.2654617,"monotonic":2349724.712475743,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.399","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.420","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.430","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.440","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.449","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.458","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.2656631,"monotonic":2349725.712676362,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.433","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.443","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.461","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.470","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:37.478","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.2664464,"monotonic":2349726.713460141,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.424","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.435","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.446","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.456","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.466","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.476","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.485","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.267223,"monotonic":2349727.71423648,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.408","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.420","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.431","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.442","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.462","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.472","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.481","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.2680006,"monotonic":2349728.7150143,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.391","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.436","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.456","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.465","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.2687998,"monotonic":2349729.715813933,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.426","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.437","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.448","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.458","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.467","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.477","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:41.486","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.269579,"monotonic":2349730.716592797,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.389","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.400","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.433","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.442","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.452","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.461","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.2703831,"monotonic":2349731.717397088,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.434","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.445","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.464","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.474","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.483","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.2711642,"monotonic":2349732.718178003,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.426","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.437","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.457","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.466","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.475","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.27197,"monotonic":2349733.718983432,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.433","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.444","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.454","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.464","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.474","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.483","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.492","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.2727625,"monotonic":2349734.719776103,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.416","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.428","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.439","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.450","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.460","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.469","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.479","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.489","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.2730267,"monotonic":2349735.720040416,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.374","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.385","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.396","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.416","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.426","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:47.444","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.2738519,"monotonic":2349736.720865601,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.354","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.365","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.377","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.397","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.406","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.415","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.424","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.274696,"monotonic":2349737.721709899,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.375","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.409","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.419","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.428","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.438","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.2755163,"monotonic":2349738.722530052,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.375","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.408","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.419","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.428","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.438","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.2760007,"monotonic":2349739.723013729,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.375","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.386","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.397","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.417","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.426","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:51.443","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.276827,"monotonic":2349740.723840845,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.392","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.424","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.07,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.434","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.443","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.452","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.461","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.2770166,"monotonic":2349741.724030315,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.376","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.397","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.408","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.417","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.426","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.444","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.2778432,"monotonic":2349742.724857021,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.378","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.389","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.400","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.420","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.428","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.2780526,"monotonic":2349743.725065985,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.375","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.409","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.419","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.428","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.438","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.2788742,"monotonic":2349744.725887854,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.379","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.390","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.402","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.423","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.432","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.2796936,"monotonic":2349745.726707438,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.383","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.417","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.427","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.2805195,"monotonic":2349746.727532797,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.384","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.396","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.407","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.417","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.427","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.2813504,"monotonic":2349747.728363884,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.416","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.427","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.438","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.449","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.468","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.478","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.487","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.2814198,"monotonic":2349748.728433408,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.362","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.373","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.384","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.394","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.404","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.413","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.421","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.430","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.2822623,"monotonic":2349749.729276162,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.379","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.390","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.400","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.411","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.421","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.430","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.439","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.2830884,"monotonic":2349750.730102261,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.383","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.394","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.405","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.425","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.433","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.442","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.451","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.283914,"monotonic":2349751.730927886,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.408","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.419","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.429","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.438","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.447","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.456","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.284734,"monotonic":2349752.731747788,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.388","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.399","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.431","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.2855494,"monotonic":2349753.732563383,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.435","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.472","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.2863526,"monotonic":2349754.733366742,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.409","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.419","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.429","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.2871795,"monotonic":2349755.734193496,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.366","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.377","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.407","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.416","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.425","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.434","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.2880235,"monotonic":2349756.73503748,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.390","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.402","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.432","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.2888458,"monotonic":2349757.735859656,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.398","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.408","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.418","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.428","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.455","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.2896729,"monotonic":2349758.736686763,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.391","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.402","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.412","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.423","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.432","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.2904937,"monotonic":2349759.737507598,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.399","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.431","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:11.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:11.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.2913218,"monotonic":2349760.738334959,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.387","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.399","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.410","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.421","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.431","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.440","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.450","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.459","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.2921426,"monotonic":2349761.739156352,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.391","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.403","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.414","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.424","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.434","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.443","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.452","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.461","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.292966,"monotonic":2349762.739979714,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.392","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.404","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.415","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.425","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.2936296,"monotonic":2349763.740642826,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.417","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.427","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.436","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:15.462","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.2944596,"monotonic":2349764.741473539,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.373","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.384","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.405","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.415","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.424","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.433","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.441","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.2953057,"monotonic":2349765.742319537,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.374","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.385","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.396","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.416","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.425","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.433","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.442","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.2961583,"monotonic":2349766.743171744,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.394","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.416","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.426","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.436","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.462","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.296983,"monotonic":2349767.743997016,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.416","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.427","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.436","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:19.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:19.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:19.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.2978086,"monotonic":2349768.744822404,"hcas":{"mlx5_0":{"port_xmit_data":380077281780531,"port_rcv_data":380480807505808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865202819938,"port_rcv_data":1400155757330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.395","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.406","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.416","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.426","gpu_util_pct":0.0,"memory_used_mib":61959.0,"power_w":77.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.436","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.462","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.2986503,"monotonic":2349769.745663867,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.436","gpu_util_pct":64.0,"memory_used_mib":70771.0,"power_w":95.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.447","gpu_util_pct":82.0,"memory_used_mib":70771.0,"power_w":85.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.458","gpu_util_pct":90.0,"memory_used_mib":70771.0,"power_w":94.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.468","gpu_util_pct":21.0,"memory_used_mib":70771.0,"power_w":105.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.477","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.486","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:21.495","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:21.504","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.2994394,"monotonic":2349770.746452893,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.395","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":100.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.406","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":92.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.417","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":98.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.427","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":102.54,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.437","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:22.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.463","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.3002658,"monotonic":2349771.747279655,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.415","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.427","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.437","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":75.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.448","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.54,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.458","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.467","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.476","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.484","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.3010132,"monotonic":2349772.748026831,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.403","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.414","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.424","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.435","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.444","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:24.462","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:24.471","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.3018382,"monotonic":2349773.74885163,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.424","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.436","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.447","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.457","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.467","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.477","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.486","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.495","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.3026383,"monotonic":2349774.749651962,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.382","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.394","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.405","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.415","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.425","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.435","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.444","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.303482,"monotonic":2349775.750495769,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.403","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.415","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.426","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.436","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.446","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.456","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.465","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:27.474","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.3042877,"monotonic":2349776.751301378,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.403","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.414","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.425","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.435","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.444","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.462","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.470","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.3050249,"monotonic":2349777.752038718,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.407","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.418","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.429","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.439","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.448","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":82.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.457","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:29.466","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:29.474","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.305847,"monotonic":2349778.752860878,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.385","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.396","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.407","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":76.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.417","gpu_util_pct":0.0,"memory_used_mib":70771.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.427","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":81.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.436","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.445","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.453","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.3059525,"monotonic":2349779.752965999,"hcas":{"mlx5_0":{"port_xmit_data":380077282039508,"port_rcv_data":380480926760996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865203078902,"port_rcv_data":1400275011543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.405","gpu_util_pct":100.0,"memory_used_mib":70771.0,"power_w":105.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.418","gpu_util_pct":19.0,"memory_used_mib":70771.0,"power_w":90.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.430","gpu_util_pct":96.0,"memory_used_mib":70771.0,"power_w":122.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.442","gpu_util_pct":100.0,"memory_used_mib":70771.0,"power_w":168.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.454","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":80.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.464","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.473","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":77.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.482","gpu_util_pct":0.0,"memory_used_mib":72577.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.3067768,"monotonic":2349780.753790648,"hcas":{"mlx5_0":{"port_xmit_data":380077397229855,"port_rcv_data":380481039653708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865318282376,"port_rcv_data":1400387893535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409654742301,"port_rcv_data":380150594509147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429533373568,"port_rcv_data":1038391588837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.408","gpu_util_pct":100.0,"memory_used_mib":71619.0,"power_w":322.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.423","gpu_util_pct":100.0,"memory_used_mib":71619.0,"power_w":293.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.434","gpu_util_pct":100.0,"memory_used_mib":71619.0,"power_w":316.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.446","gpu_util_pct":100.0,"memory_used_mib":71619.0,"power_w":344.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.457","gpu_util_pct":100.0,"memory_used_mib":81725.0,"power_w":216.54,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.468","gpu_util_pct":100.0,"memory_used_mib":81725.0,"power_w":127.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.478","gpu_util_pct":100.0,"memory_used_mib":81725.0,"power_w":143.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.488","gpu_util_pct":100.0,"memory_used_mib":81725.0,"power_w":139.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.3076115,"monotonic":2349781.754624396,"hcas":{"mlx5_0":{"port_xmit_data":380077397229855,"port_rcv_data":380481039653708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865318282376,"port_rcv_data":1400387893535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409710566929,"port_rcv_data":380150595568756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429584195128,"port_rcv_data":1038392465207,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.433","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":191.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.449","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":187.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.461","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":190.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.471","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":162.06,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.480","gpu_util_pct":100.0,"memory_used_mib":82679.0,"power_w":232.0,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.490","gpu_util_pct":100.0,"memory_used_mib":82679.0,"power_w":235.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.500","gpu_util_pct":100.0,"memory_used_mib":82679.0,"power_w":229.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.510","gpu_util_pct":100.0,"memory_used_mib":82679.0,"power_w":232.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.3084214,"monotonic":2349782.755434083,"hcas":{"mlx5_0":{"port_xmit_data":380077513283062,"port_rcv_data":380481039778733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434334760,"port_rcv_data":1400388014953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409822738002,"port_rcv_data":380150597728896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429706300408,"port_rcv_data":1038394734981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.415","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.431","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":79.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.442","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.453","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.463","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":192.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.472","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":207.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:34.482","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":196.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:34.491","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":203.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.309244,"monotonic":2349783.756256914,"hcas":{"mlx5_0":{"port_xmit_data":380077513283277,"port_rcv_data":380481039778780,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434334975,"port_rcv_data":1400388014999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379409954378973,"port_rcv_data":380150600054501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429830496323,"port_rcv_data":1038396993362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.410","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":79.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.425","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.437","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.447","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.457","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":81.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.467","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":79.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.476","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":77.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.486","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":79.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.3100688,"monotonic":2349784.757081757,"hcas":{"mlx5_0":{"port_xmit_data":380077513285532,"port_rcv_data":380481039779369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434337768,"port_rcv_data":1400388015650,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410095739089,"port_rcv_data":380150602446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1429970068676,"port_rcv_data":1038399516514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.451","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.466","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.477","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.488","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.43,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.499","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":80.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.509","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:36.519","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":76.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:36.528","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.3108456,"monotonic":2349785.757858903,"hcas":{"mlx5_0":{"port_xmit_data":380077513287441,"port_rcv_data":380481039779896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434340199,"port_rcv_data":1400388016239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410194636069,"port_rcv_data":380150604241655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430073265432,"port_rcv_data":1038401307229,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.447","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.461","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.472","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.483","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.493","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":80.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.503","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.512","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":76.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.521","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.311631,"monotonic":2349786.758644427,"hcas":{"mlx5_0":{"port_xmit_data":380077513287742,"port_rcv_data":380481039779973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434340484,"port_rcv_data":1400388016317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.464","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.478","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.489","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.499","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.67,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.510","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":81.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.519","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.528","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":76.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.537","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.3120363,"monotonic":2349787.759049283,"hcas":{"mlx5_0":{"port_xmit_data":380077513287742,"port_rcv_data":380481039779973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434340484,"port_rcv_data":1400388016317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.457","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.469","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.480","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.491","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.501","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":81.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.511","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.520","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":77.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.529","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.3128116,"monotonic":2349788.759825013,"hcas":{"mlx5_0":{"port_xmit_data":380077513287742,"port_rcv_data":380481039779973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434340484,"port_rcv_data":1400388016317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.432","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":79.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.444","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.455","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.466","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.92,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.475","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":81.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.485","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:40.494","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.503","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.3136144,"monotonic":2349789.760628152,"hcas":{"mlx5_0":{"port_xmit_data":380077513287742,"port_rcv_data":380481039779973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434340484,"port_rcv_data":1400388016317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.393","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.405","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.416","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.427","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.437","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":80.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:41.446","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.455","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":76.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.465","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.3144689,"monotonic":2349790.76148277,"hcas":{"mlx5_0":{"port_xmit_data":380077513287742,"port_rcv_data":380481039779973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434340484,"port_rcv_data":1400388016317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.394","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.405","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.417","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.427","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:42.438","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.447","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.456","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":76.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.465","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.3153086,"monotonic":2349791.762322344,"hcas":{"mlx5_0":{"port_xmit_data":380077513287742,"port_rcv_data":380481039779973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434340484,"port_rcv_data":1400388016317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.394","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":78.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.406","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.417","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.428","gpu_util_pct":0.0,"memory_used_mib":71619.0,"power_w":76.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.438","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":80.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.447","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.457","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":76.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.466","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.3161511,"monotonic":2349792.763164825,"hcas":{"mlx5_0":{"port_xmit_data":380077513630760,"port_rcv_data":380481159118706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865434688937,"port_rcv_data":1400507359926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.417","gpu_util_pct":93.0,"memory_used_mib":71731.0,"power_w":103.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.430","gpu_util_pct":100.0,"memory_used_mib":71731.0,"power_w":104.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.441","gpu_util_pct":100.0,"memory_used_mib":71731.0,"power_w":101.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.452","gpu_util_pct":100.0,"memory_used_mib":71731.0,"power_w":109.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.464","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":81.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.473","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.482","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":76.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.491","gpu_util_pct":0.0,"memory_used_mib":82679.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.3169653,"monotonic":2349793.763979384,"hcas":{"mlx5_0":{"port_xmit_data":380077628772916,"port_rcv_data":380481271959088,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865549812913,"port_rcv_data":1400620157879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410211766554,"port_rcv_data":380150604541394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430090395900,"port_rcv_data":1038401604674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.421","gpu_util_pct":100.0,"memory_used_mib":71731.0,"power_w":257.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.433","gpu_util_pct":100.0,"memory_used_mib":71731.0,"power_w":297.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.446","gpu_util_pct":100.0,"memory_used_mib":71731.0,"power_w":268.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.458","gpu_util_pct":100.0,"memory_used_mib":71731.0,"power_w":276.92,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:45.468","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":132.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:45.479","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":126.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:45.490","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":128.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.502","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":131.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.3177905,"monotonic":2349794.764803441,"hcas":{"mlx5_0":{"port_xmit_data":380077629011283,"port_rcv_data":380481391194223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865550052805,"port_rcv_data":1400739393469,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410247133167,"port_rcv_data":380150605248769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430125762300,"port_rcv_data":1038402300639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.458","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":289.22,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.473","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":309.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.484","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":279.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.494","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":292.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.505","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":247.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.516","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":230.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.526","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":253.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.537","gpu_util_pct":100.0,"memory_used_mib":82791.0,"power_w":263.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.318584,"monotonic":2349795.765596905,"hcas":{"mlx5_0":{"port_xmit_data":380077744208284,"port_rcv_data":380481510620134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865665244657,"port_rcv_data":1400858814845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410386125397,"port_rcv_data":380150607810150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430263613740,"port_rcv_data":1038404818893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.427","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":242.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.443","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":290.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.455","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":230.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.465","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":213.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.475","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":292.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.486","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":281.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.495","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":315.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.506","gpu_util_pct":93.0,"memory_used_mib":83575.0,"power_w":326.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.319404,"monotonic":2349796.766416871,"hcas":{"mlx5_0":{"port_xmit_data":380077859256083,"port_rcv_data":380481569377316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865780278213,"port_rcv_data":1400917537618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410512382080,"port_rcv_data":380150610346125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430388790060,"port_rcv_data":1038407300102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.426","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":275.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.442","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":293.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.455","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":278.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.466","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":268.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:48.476","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":282.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.485","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":280.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.495","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":265.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.504","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":272.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.3202233,"monotonic":2349797.767236982,"hcas":{"mlx5_0":{"port_xmit_data":380077859595399,"port_rcv_data":380481688711292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865780617047,"port_rcv_data":1401036870910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410636164791,"port_rcv_data":380150612801309,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430514792700,"port_rcv_data":1038409854223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.468","gpu_util_pct":99.0,"memory_used_mib":72403.0,"power_w":190.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.484","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":205.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.496","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":196.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.507","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":197.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.519","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":313.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.528","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":280.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.538","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":319.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.547","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":327.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.3210044,"monotonic":2349798.768018068,"hcas":{"mlx5_0":{"port_xmit_data":380077974801347,"port_rcv_data":380481808144608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1865895816619,"port_rcv_data":1401156295985,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410768789361,"port_rcv_data":380150615277126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430650846300,"port_rcv_data":1038412313531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.463","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":293.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.478","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":266.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.488","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":283.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.499","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":297.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.509","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":285.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.519","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":249.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.529","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":263.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.539","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":269.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.3217843,"monotonic":2349799.7687975,"hcas":{"mlx5_0":{"port_xmit_data":380078089713297,"port_rcv_data":380481819305252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866010729449,"port_rcv_data":1401167448897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379410892572364,"port_rcv_data":380150617615646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430771199100,"port_rcv_data":1038414716248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.461","gpu_util_pct":0.0,"memory_used_mib":72403.0,"power_w":266.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.476","gpu_util_pct":0.0,"memory_used_mib":72403.0,"power_w":242.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.487","gpu_util_pct":0.0,"memory_used_mib":72403.0,"power_w":260.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:51.497","gpu_util_pct":0.0,"memory_used_mib":72403.0,"power_w":258.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:51.507","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":274.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.516","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":286.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.526","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":263.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.535","gpu_util_pct":100.0,"memory_used_mib":83575.0,"power_w":272.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.3225698,"monotonic":2349800.769583156,"hcas":{"mlx5_0":{"port_xmit_data":380078090036221,"port_rcv_data":380481938623590,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866011048321,"port_rcv_data":1401286762879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379411016355107,"port_rcv_data":380150619950574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1430894258305,"port_rcv_data":1038416981681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.428","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":114.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.445","gpu_util_pct":99.0,"memory_used_mib":72403.0,"power_w":115.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.456","gpu_util_pct":98.0,"memory_used_mib":72403.0,"power_w":112.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.467","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":132.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.477","gpu_util_pct":60.0,"memory_used_mib":83577.0,"power_w":248.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.487","gpu_util_pct":59.0,"memory_used_mib":83577.0,"power_w":280.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.496","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":248.4,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.505","gpu_util_pct":98.0,"memory_used_mib":83577.0,"power_w":256.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.323382,"monotonic":2349801.770395931,"hcas":{"mlx5_0":{"port_xmit_data":380078321199286,"port_rcv_data":380482052625959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866242211753,"port_rcv_data":1401400735961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379411140142823,"port_rcv_data":380150622187384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1431016632820,"port_rcv_data":1038419167910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.441","gpu_util_pct":97.0,"memory_used_mib":72403.0,"power_w":218.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.452","gpu_util_pct":99.0,"memory_used_mib":72403.0,"power_w":246.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.461","gpu_util_pct":99.0,"memory_used_mib":72403.0,"power_w":231.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.469","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":239.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.478","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":117.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.486","gpu_util_pct":25.0,"memory_used_mib":83577.0,"power_w":121.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.494","gpu_util_pct":10.0,"memory_used_mib":83577.0,"power_w":107.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.502","gpu_util_pct":75.0,"memory_used_mib":83577.0,"power_w":129.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.3235395,"monotonic":2349802.770552902,"hcas":{"mlx5_0":{"port_xmit_data":380078321533487,"port_rcv_data":380482171956597,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866242526675,"port_rcv_data":1401520047001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379411277783373,"port_rcv_data":380150624848616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1431151392580,"port_rcv_data":1038421826082,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.463","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":328.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.475","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":360.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.485","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":336.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.494","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":313.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.503","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":324.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.511","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":303.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.520","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":290.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.527","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":299.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.324342,"monotonic":2349803.771355508,"hcas":{"mlx5_0":{"port_xmit_data":380078436754636,"port_rcv_data":380482291407181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866357739459,"port_rcv_data":1401639489531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379411431916471,"port_rcv_data":380150627881316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1431307687540,"port_rcv_data":1038424806864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.429","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":254.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.440","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":267.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.449","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":260.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.458","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":249.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:55.467","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":309.93,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.475","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":336.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.482","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":324.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.490","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":323.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.325188,"monotonic":2349804.772200748,"hcas":{"mlx5_0":{"port_xmit_data":380078624807748,"port_rcv_data":380482351604061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866527746563,"port_rcv_data":1401699627788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379411588382223,"port_rcv_data":380150630959743,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1431464819401,"port_rcv_data":1038427871663,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.447","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":315.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.461","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":287.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.470","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":313.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.479","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":337.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.486","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":274.53,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.494","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":279.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.501","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":263.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.508","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":263.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.3257527,"monotonic":2349805.772765768,"hcas":{"mlx5_0":{"port_xmit_data":380078625098294,"port_rcv_data":380482470889188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866528085863,"port_rcv_data":1401818960971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379411734383674,"port_rcv_data":380150633596382,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1431604264346,"port_rcv_data":1038430466410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.427","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":222.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.439","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":225.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.446","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":220.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.454","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":221.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.462","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":328.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.469","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":305.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.477","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":308.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.484","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":338.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.3265934,"monotonic":2349806.773607233,"hcas":{"mlx5_0":{"port_xmit_data":380078740275767,"port_rcv_data":380482588835548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866643261276,"port_rcv_data":1401936891823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379411884569105,"port_rcv_data":380150636464603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1431763191782,"port_rcv_data":1038433519652,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.464","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":293.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.476","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":267.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.485","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":292.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.495","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":289.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:58.502","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":238.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:58.510","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":246.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:58.517","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":221.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:58.524","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":228.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.327408,"monotonic":2349807.77442121,"hcas":{"mlx5_0":{"port_xmit_data":380078758845937,"port_rcv_data":380482708202842,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866661837037,"port_rcv_data":1402056263533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379412038501064,"port_rcv_data":380150639382171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1431911776456,"port_rcv_data":1038436324067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.447","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":301.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.458","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":321.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.467","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":295.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.475","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":284.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.484","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":300.2,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.491","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":255.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.500","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":255.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.507","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":283.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.3282444,"monotonic":2349808.775257642,"hcas":{"mlx5_0":{"port_xmit_data":380078874013250,"port_rcv_data":380482824571475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866777005239,"port_rcv_data":1402172591412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379412188858684,"port_rcv_data":380150642302347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1432070605816,"port_rcv_data":1038439420657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.412","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":235.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.424","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":238.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.432","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":237.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.441","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":227.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.449","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":334.34,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.458","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":323.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.465","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":316.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.472","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":334.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.3290355,"monotonic":2349809.776048614,"hcas":{"mlx5_0":{"port_xmit_data":380079010196919,"port_rcv_data":380482944044070,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866877164692,"port_rcv_data":1402292007370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379412349328070,"port_rcv_data":380150645454645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1432222341304,"port_rcv_data":1038442487762,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.469","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":333.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.483","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":300.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.493","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":327.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.503","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":330.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.511","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":284.41,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.518","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":284.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:01.526","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":277.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.533","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":315.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.3298457,"monotonic":2349810.776858978,"hcas":{"mlx5_0":{"port_xmit_data":380079125384379,"port_rcv_data":380483063459952,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1866992356308,"port_rcv_data":1402411426502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379412525939686,"port_rcv_data":380150648978959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1432404560195,"port_rcv_data":1038446069626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.429","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":277.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.440","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":241.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.449","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":284.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.459","gpu_util_pct":100.0,"memory_used_mib":72403.0,"power_w":253.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.468","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":333.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.476","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":290.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.486","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":288.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.493","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":320.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.3306882,"monotonic":2349811.777701659,"hcas":{"mlx5_0":{"port_xmit_data":380079240270283,"port_rcv_data":380483064932115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867107240594,"port_rcv_data":1402412864009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379412696487930,"port_rcv_data":380150652258080,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1432569488927,"port_rcv_data":1038449149445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.427","gpu_util_pct":0.0,"memory_used_mib":72405.0,"power_w":203.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.438","gpu_util_pct":0.0,"memory_used_mib":72405.0,"power_w":218.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.446","gpu_util_pct":0.0,"memory_used_mib":72405.0,"power_w":207.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.454","gpu_util_pct":0.0,"memory_used_mib":72405.0,"power_w":181.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.461","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":250.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.471","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":239.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.478","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":259.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.485","gpu_util_pct":100.0,"memory_used_mib":83577.0,"power_w":259.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.3315532,"monotonic":2349812.778566395,"hcas":{"mlx5_0":{"port_xmit_data":380079315391439,"port_rcv_data":380483065016137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867163712391,"port_rcv_data":1402412929341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379412873286491,"port_rcv_data":380150655468930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1432744371252,"port_rcv_data":1038452438871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.473","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.484","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.498","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.505","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":227.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.511","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":188.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.517","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":247.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.523","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":239.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.3323662,"monotonic":2349813.779379685,"hcas":{"mlx5_0":{"port_xmit_data":380079602625576,"port_rcv_data":380483065334525,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377144623,"port_rcv_data":1402413170211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379413036148951,"port_rcv_data":380150658428547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1432914765084,"port_rcv_data":1038455546342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.431","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.442","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.450","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.465","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":84.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:05.472","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.479","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.486","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.3332148,"monotonic":2349814.780228015,"hcas":{"mlx5_0":{"port_xmit_data":380079602627947,"port_rcv_data":380483065335036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377146994,"port_rcv_data":1402413170723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379413211667236,"port_rcv_data":380150661504041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1433090283193,"port_rcv_data":1038458615280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.431","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.441","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.449","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.464","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.472","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.479","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.486","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.3336504,"monotonic":2349815.780663918,"hcas":{"mlx5_0":{"port_xmit_data":380079602629704,"port_rcv_data":380483065335517,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377148623,"port_rcv_data":1402413171203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379413375440691,"port_rcv_data":380150664313013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1433248533448,"port_rcv_data":1038461385982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.431","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.441","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.449","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.465","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.472","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.479","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.486","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.3345134,"monotonic":2349816.781526698,"hcas":{"mlx5_0":{"port_xmit_data":380079602630351,"port_rcv_data":380483065335656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377149270,"port_rcv_data":1402413171343,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379413547163674,"port_rcv_data":380150667413091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1433425779503,"port_rcv_data":1038464531428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.431","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.441","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.449","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:08.464","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.472","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.479","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.485","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.3353646,"monotonic":2349817.782378143,"hcas":{"mlx5_0":{"port_xmit_data":380079602632065,"port_rcv_data":380483065336168,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377152140,"port_rcv_data":1402413171978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379413717911564,"port_rcv_data":380150670324223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1433588746304,"port_rcv_data":1038467316437,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.448","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.458","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.466","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.474","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.23,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.481","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.488","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.495","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.501","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.336195,"monotonic":2349818.783208819,"hcas":{"mlx5_0":{"port_xmit_data":380079602633568,"port_rcv_data":380483065336555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377153563,"port_rcv_data":1402413172366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379413885091561,"port_rcv_data":380150673302967,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1433761594186,"port_rcv_data":1038470337945,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.467","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.478","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.486","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.494","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.501","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.508","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.514","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.521","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.3370183,"monotonic":2349819.784032169,"hcas":{"mlx5_0":{"port_xmit_data":380079602635453,"port_rcv_data":380483065337036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377155856,"port_rcv_data":1402413172893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414050304166,"port_rcv_data":380150676204443,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1433928918287,"port_rcv_data":1038473302367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.477","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.485","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.490","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.496","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.502","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.508","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:11.513","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.519","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.3378527,"monotonic":2349820.784865947,"hcas":{"mlx5_0":{"port_xmit_data":380079602636807,"port_rcv_data":380483065337392,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377157795,"port_rcv_data":1402413173311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414226928395,"port_rcv_data":380150679281053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434105541914,"port_rcv_data":1038476398352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.435","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.445","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.453","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.461","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:12.468","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:12.475","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.482","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.488","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.3387108,"monotonic":2349821.785724146,"hcas":{"mlx5_0":{"port_xmit_data":380079602639462,"port_rcv_data":380483065337981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377160597,"port_rcv_data":1402413173916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414394255028,"port_rcv_data":380150682196913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434271558785,"port_rcv_data":1038479265914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.472","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.482","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.498","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.505","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.513","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.519","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.526","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.3395464,"monotonic":2349822.78655946,"hcas":{"mlx5_0":{"port_xmit_data":380079602642734,"port_rcv_data":380483065338725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377163821,"port_rcv_data":1402413174660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414529335607,"port_rcv_data":380150684559997,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434407948580,"port_rcv_data":1038481688781,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.438","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.449","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.465","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.472","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.479","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.485","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.492","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.340402,"monotonic":2349823.787415163,"hcas":{"mlx5_0":{"port_xmit_data":380079602644406,"port_rcv_data":380483065339175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377165381,"port_rcv_data":1402413175109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414647768595,"port_rcv_data":380150686553669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434520885575,"port_rcv_data":1038483650182,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.475","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.485","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.493","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.501","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.509","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:15.516","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:15.522","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.529","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.3410192,"monotonic":2349824.788032415,"hcas":{"mlx5_0":{"port_xmit_data":380079602645909,"port_rcv_data":380483065339562,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377166804,"port_rcv_data":1402413175497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414766745285,"port_rcv_data":380150688682067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434645358098,"port_rcv_data":1038485809642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.439","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.449","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.465","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.473","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.480","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.487","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.493","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.34187,"monotonic":2349825.788883581,"hcas":{"mlx5_0":{"port_xmit_data":380079602649118,"port_rcv_data":380483065340415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377170812,"port_rcv_data":1402413176442,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414877402186,"port_rcv_data":380150690548081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434756014040,"port_rcv_data":1038487678787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.438","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.449","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.465","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.472","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.479","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.486","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.492","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.3427224,"monotonic":2349826.789735945,"hcas":{"mlx5_0":{"port_xmit_data":380079602650483,"port_rcv_data":380483065340942,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377172718,"port_rcv_data":1402413177047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414951865178,"port_rcv_data":380150691796792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434830476379,"port_rcv_data":1038488922182,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.475","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.485","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.493","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.501","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.508","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.515","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.522","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.528","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.3435369,"monotonic":2349827.790550501,"hcas":{"mlx5_0":{"port_xmit_data":380079602650759,"port_rcv_data":380483065341066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377173580,"port_rcv_data":1402413177233,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379414987784216,"port_rcv_data":380150692412715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434866394895,"port_rcv_data":1038489540694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.475","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.485","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.493","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.501","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.508","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.515","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.522","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.528","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.34436,"monotonic":2349828.791373474,"hcas":{"mlx5_0":{"port_xmit_data":380079602651035,"port_rcv_data":380483065341190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377174442,"port_rcv_data":1402413177419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415023288810,"port_rcv_data":380150693020780,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434901898967,"port_rcv_data":1038490149890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.479","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.488","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.496","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.503","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.511","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:20.518","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.524","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.531","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.3451738,"monotonic":2349829.792187422,"hcas":{"mlx5_0":{"port_xmit_data":380079602651311,"port_rcv_data":380483065341314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377175304,"port_rcv_data":1402413177605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415058655256,"port_rcv_data":380150693624722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434937264891,"port_rcv_data":1038490758715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.483","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.500","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.507","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.11,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.515","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.522","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.529","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.535","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.346012,"monotonic":2349830.79302575,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.493","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.505","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.516","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.526","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.13,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.536","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:22.545","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.554","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.563","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.3468018,"monotonic":2349831.793815258,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.493","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.504","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.515","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.525","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.535","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.544","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.553","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.562","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.3475857,"monotonic":2349832.794599612,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.473","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.485","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.496","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.506","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:24.516","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.525","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.534","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.544","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.3483891,"monotonic":2349833.795402722,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.497","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.508","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.530","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.540","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.549","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.559","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.567","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.3490212,"monotonic":2349834.7960347,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.503","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.514","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.525","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:26.535","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.544","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.553","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.563","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.349821,"monotonic":2349835.796834811,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.496","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.506","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.515","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.524","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.532","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.540","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.548","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.555","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.3506143,"monotonic":2349836.797628153,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.446","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.455","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:28.464","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:28.473","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:28.481","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:28.489","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.497","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.504","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.351456,"monotonic":2349837.798469826,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.429","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.438","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.447","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.456","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.464","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.472","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.480","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.488","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.3523128,"monotonic":2349838.799326672,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.470","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.480","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.489","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.498","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:30.507","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:30.515","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.523","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.532","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.3531272,"monotonic":2349839.800140892,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.449","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.459","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.468","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.477","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.485","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.493","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.502","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.510","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.3533545,"monotonic":2349840.800368327,"hcas":{"mlx5_0":{"port_xmit_data":380079602651932,"port_rcv_data":380483065341593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867377177243,"port_rcv_data":1402413178023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.454","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.464","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.474","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:32.484","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.13,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:32.495","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.504","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.513","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.523","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.3541758,"monotonic":2349841.801189576,"hcas":{"mlx5_0":{"port_xmit_data":380079718106286,"port_rcv_data":380483289921289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867492639990,"port_rcv_data":1402637761628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070259919,"port_rcv_data":380150693797764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.454","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":172.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.466","gpu_util_pct":61.0,"memory_used_mib":72407.0,"power_w":178.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.476","gpu_util_pct":48.0,"memory_used_mib":72407.0,"power_w":173.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.485","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":185.94,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.494","gpu_util_pct":96.0,"memory_used_mib":83581.0,"power_w":99.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.502","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.512","gpu_util_pct":87.0,"memory_used_mib":83581.0,"power_w":81.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.521","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":85.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.35493,"monotonic":2349842.801943496,"hcas":{"mlx5_0":{"port_xmit_data":380079718132054,"port_rcv_data":380483302505470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867492667326,"port_rcv_data":1402650344134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415070260132,"port_rcv_data":380150693798029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1434948868380,"port_rcv_data":1038490931307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.454","gpu_util_pct":98.0,"memory_used_mib":72407.0,"power_w":310.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.464","gpu_util_pct":78.0,"memory_used_mib":72407.0,"power_w":299.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.472","gpu_util_pct":92.0,"memory_used_mib":72407.0,"power_w":305.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.480","gpu_util_pct":16.0,"memory_used_mib":72407.0,"power_w":281.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:34.487","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":250.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.495","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":242.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.502","gpu_util_pct":73.0,"memory_used_mib":83581.0,"power_w":257.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.509","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":253.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.3557806,"monotonic":2349843.802793965,"hcas":{"mlx5_0":{"port_xmit_data":380079718132054,"port_rcv_data":380483302505470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867492667326,"port_rcv_data":1402650344134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415229409243,"port_rcv_data":380150696891820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435107085260,"port_rcv_data":1038494033012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.455","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.465","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.473","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.481","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.488","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":303.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.495","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":321.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.502","gpu_util_pct":2.0,"memory_used_mib":83581.0,"power_w":291.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.509","gpu_util_pct":72.0,"memory_used_mib":83581.0,"power_w":301.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.3566399,"monotonic":2349844.803652737,"hcas":{"mlx5_0":{"port_xmit_data":380079858169861,"port_rcv_data":380483302652181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595505519,"port_rcv_data":1402650454773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415378542338,"port_rcv_data":380150699494378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435255334068,"port_rcv_data":1038496605687,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.492","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.502","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.510","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.94,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.526","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.533","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.540","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.547","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.3572092,"monotonic":2349845.804221983,"hcas":{"mlx5_0":{"port_xmit_data":380079858172286,"port_rcv_data":380483302652832,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595508450,"port_rcv_data":1402650455486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415532583041,"port_rcv_data":380150702160239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435411190440,"port_rcv_data":1038499330726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.492","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.502","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.511","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.526","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.534","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.541","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.548","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.3577707,"monotonic":2349846.804783948,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.471","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.482","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:38.499","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:38.507","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.515","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.523","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.530","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.3586054,"monotonic":2349847.805618189,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.498","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.509","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.520","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.531","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.09,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.540","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.549","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.558","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.567","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.3593946,"monotonic":2349848.806408256,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.486","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.497","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.507","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.518","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.528","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.537","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.546","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.555","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.3601992,"monotonic":2349849.807212718,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.484","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.495","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:41.506","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:41.516","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.526","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.535","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.544","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.553","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.3610117,"monotonic":2349850.808025406,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.512","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.526","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.537","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.548","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.558","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.568","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.577","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.586","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.361792,"monotonic":2349851.808805196,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.506","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.517","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.528","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:43.539","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:43.549","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.559","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.568","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.577","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.3625803,"monotonic":2349852.809593801,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.502","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.514","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.525","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.535","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.545","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.554","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.564","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.572","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.3633773,"monotonic":2349853.810390887,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.508","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.520","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.531","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.542","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.552","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.561","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.570","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.579","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.3641593,"monotonic":2349854.811172918,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.507","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.530","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.541","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.550","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.560","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.569","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.578","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.3649619,"monotonic":2349855.81197564,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.490","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.501","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.512","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.523","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.533","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.543","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.552","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.561","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.3650186,"monotonic":2349856.812032307,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.500","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.512","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.523","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:48.533","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:48.543","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.552","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.561","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.570","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.365811,"monotonic":2349857.812824895,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.445","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.457","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:49.468","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.479","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.489","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.498","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.508","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.517","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.3666553,"monotonic":2349858.813669125,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.484","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.495","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.506","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.517","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.526","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.536","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.545","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.554","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.367459,"monotonic":2349859.814472857,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.446","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.458","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.469","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.479","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.489","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.498","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.508","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.516","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.368304,"monotonic":2349860.815317843,"hcas":{"mlx5_0":{"port_xmit_data":380079858174240,"port_rcv_data":380483302653344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595510958,"port_rcv_data":1402650456060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.503","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.514","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.525","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.535","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.545","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.554","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.563","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.3691037,"monotonic":2349861.816117372,"hcas":{"mlx5_0":{"port_xmit_data":380079858215493,"port_rcv_data":380483316858773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867595560155,"port_rcv_data":1402666870031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.468","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.482","gpu_util_pct":61.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.494","gpu_util_pct":1.0,"memory_used_mib":72407.0,"power_w":77.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.506","gpu_util_pct":66.0,"memory_used_mib":72407.0,"power_w":98.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.517","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.526","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.535","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.544","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.3699243,"monotonic":2349862.81693814,"hcas":{"mlx5_0":{"port_xmit_data":380079973656284,"port_rcv_data":380483539867797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867710977910,"port_rcv_data":1402887636817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415640546475,"port_rcv_data":380150704001050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435519153240,"port_rcv_data":1038501176668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.470","gpu_util_pct":31.0,"memory_used_mib":72407.0,"power_w":260.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.483","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":281.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:54.497","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":251.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.507","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":264.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.520","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":124.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.531","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":121.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.540","gpu_util_pct":92.0,"memory_used_mib":83581.0,"power_w":99.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.550","gpu_util_pct":90.0,"memory_used_mib":83581.0,"power_w":103.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.370734,"monotonic":2349863.817747729,"hcas":{"mlx5_0":{"port_xmit_data":380079973656284,"port_rcv_data":380483539867797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867710977910,"port_rcv_data":1402887636817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415663837488,"port_rcv_data":380150704415723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435535553640,"port_rcv_data":1038501282781,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.486","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":256.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.499","gpu_util_pct":36.0,"memory_used_mib":72407.0,"power_w":284.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.509","gpu_util_pct":50.0,"memory_used_mib":72407.0,"power_w":242.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.520","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":242.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.529","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":237.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.540","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":252.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.548","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":248.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.559","gpu_util_pct":39.0,"memory_used_mib":83581.0,"power_w":262.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.3715582,"monotonic":2349864.818570664,"hcas":{"mlx5_0":{"port_xmit_data":380080052766995,"port_rcv_data":380483539951470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867755471745,"port_rcv_data":1402887686200,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415790856377,"port_rcv_data":380150706923469,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435669463060,"port_rcv_data":1038504072021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.475","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.490","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.500","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.511","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.524","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":275.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.535","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":289.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.546","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":309.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.556","gpu_util_pct":38.0,"memory_used_mib":83581.0,"power_w":313.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.3723779,"monotonic":2349865.81939102,"hcas":{"mlx5_0":{"port_xmit_data":380080113694006,"port_rcv_data":380483540015361,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813816033,"port_rcv_data":1402887747595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379415905937546,"port_rcv_data":380150708959549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435784543997,"port_rcv_data":1038506129476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.475","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.489","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.500","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.511","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.03,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.521","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.530","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.540","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.549","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.3732114,"monotonic":2349866.82022417,"hcas":{"mlx5_0":{"port_xmit_data":380080113695607,"port_rcv_data":380483540015764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813817895,"port_rcv_data":1402887748029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416030110489,"port_rcv_data":380150711108066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1435906055463,"port_rcv_data":1038508182187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.515","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.528","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.539","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.559","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.569","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.578","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.587","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.3740072,"monotonic":2349867.821020145,"hcas":{"mlx5_0":{"port_xmit_data":380080113697362,"port_rcv_data":380483540016229,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813820042,"port_rcv_data":1402887748541,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416153722272,"port_rcv_data":380150713213013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436034938674,"port_rcv_data":1038510501328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.499","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.512","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.523","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.534","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.544","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.553","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.563","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.572","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.3748097,"monotonic":2349868.821823695,"hcas":{"mlx5_0":{"port_xmit_data":380080113698470,"port_rcv_data":380483540016555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813821542,"port_rcv_data":1402887748913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.458","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.469","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.480","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.490","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.500","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.509","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.518","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.527","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.375654,"monotonic":2349869.822667585,"hcas":{"mlx5_0":{"port_xmit_data":380080113698470,"port_rcv_data":380483540016555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813821542,"port_rcv_data":1402887748913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.520","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.531","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.542","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:01.551","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.559","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.566","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.573","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.581","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.376451,"monotonic":2349870.823464934,"hcas":{"mlx5_0":{"port_xmit_data":380080113698470,"port_rcv_data":380483540016555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813821542,"port_rcv_data":1402887748913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.476","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.488","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.499","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.510","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.520","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.529","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.539","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.547","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.377029,"monotonic":2349871.824042631,"hcas":{"mlx5_0":{"port_xmit_data":380080113698470,"port_rcv_data":380483540016555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813821542,"port_rcv_data":1402887748913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.456","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.468","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.479","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:03.490","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.500","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.509","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.518","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.527","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.3778758,"monotonic":2349872.824889342,"hcas":{"mlx5_0":{"port_xmit_data":380080113698470,"port_rcv_data":380483540016555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813821542,"port_rcv_data":1402887748913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.475","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.487","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.498","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.509","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.519","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.528","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.538","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.547","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.3786998,"monotonic":2349873.825713425,"hcas":{"mlx5_0":{"port_xmit_data":380080113698470,"port_rcv_data":380483540016555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867813821542,"port_rcv_data":1402887748913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.476","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.487","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.498","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.509","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.518","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.528","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.537","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.546","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.3795295,"monotonic":2349874.826543256,"hcas":{"mlx5_0":{"port_xmit_data":380080114030099,"port_rcv_data":380483659343665,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867814168620,"port_rcv_data":1403007091395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.495","gpu_util_pct":93.0,"memory_used_mib":72407.0,"power_w":107.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.508","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":123.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.521","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":125.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.532","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":112.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.543","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.552","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.561","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.570","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.3803322,"monotonic":2349875.827346106,"hcas":{"mlx5_0":{"port_xmit_data":380080229184674,"port_rcv_data":380483775759015,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867929335281,"port_rcv_data":1403123490292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416210832933,"port_rcv_data":380150714228991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436089438100,"port_rcv_data":1038511439605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.488","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":286.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.500","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":264.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.511","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":297.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.522","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":286.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.532","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":140.82,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.543","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":147.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.553","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":133.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.563","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":136.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.3811452,"monotonic":2349876.828158846,"hcas":{"mlx5_0":{"port_xmit_data":380080229184674,"port_rcv_data":380483775759015,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1867929335281,"port_rcv_data":1403123490292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416246199643,"port_rcv_data":380150714946942,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436124804500,"port_rcv_data":1038512143197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.520","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":228.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.533","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":234.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.544","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":196.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.554","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":195.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.564","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":241.65,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.575","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":253.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.584","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":234.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.595","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":249.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.3816824,"monotonic":2349877.828695411,"hcas":{"mlx5_0":{"port_xmit_data":380080367126975,"port_rcv_data":380483775906438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030677365,"port_rcv_data":1403123600001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416381348510,"port_rcv_data":380150717381271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436251387708,"port_rcv_data":1038514416101,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.504","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.518","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:09.528","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:09.539","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.549","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":226.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.558","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":214.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.567","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":219.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.576","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":227.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.3824935,"monotonic":2349878.829506772,"hcas":{"mlx5_0":{"port_xmit_data":380080367127191,"port_rcv_data":380483775906485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030677580,"port_rcv_data":1403123600047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416511246369,"port_rcv_data":380150719824133,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436389849951,"port_rcv_data":1038517020442,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.490","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.503","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.511","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.527","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.534","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.542","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.550","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.3833365,"monotonic":2349879.830349027,"hcas":{"mlx5_0":{"port_xmit_data":380080367129446,"port_rcv_data":380483775907074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030680373,"port_rcv_data":1403123600698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416635235120,"port_rcv_data":380150721986708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436513838052,"port_rcv_data":1038519178631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.486","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.501","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.512","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.523","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.533","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.542","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.552","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.561","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.3841875,"monotonic":2349880.831199655,"hcas":{"mlx5_0":{"port_xmit_data":380080367130985,"port_rcv_data":380483775907492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030682304,"port_rcv_data":1403123601163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416754581023,"port_rcv_data":380150723998856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436633867738,"port_rcv_data":1038521261242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.523","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.537","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.547","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.558","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.569","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.579","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.589","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.598","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.3849757,"monotonic":2349881.831988408,"hcas":{"mlx5_0":{"port_xmit_data":380080367131655,"port_rcv_data":380483775907678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030683089,"port_rcv_data":1403123601365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.506","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.518","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:13.529","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.539","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.78,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.549","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.558","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.567","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.576","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.3852518,"monotonic":2349882.832265517,"hcas":{"mlx5_0":{"port_xmit_data":380080367131655,"port_rcv_data":380483775907678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030683089,"port_rcv_data":1403123601365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.525","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.536","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.548","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.558","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.568","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.578","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.587","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.3860238,"monotonic":2349883.833036534,"hcas":{"mlx5_0":{"port_xmit_data":380080367131655,"port_rcv_data":380483775907678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030683089,"port_rcv_data":1403123601365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.503","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.514","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.526","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.536","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.546","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.556","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.565","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.386835,"monotonic":2349884.833849005,"hcas":{"mlx5_0":{"port_xmit_data":380080367131655,"port_rcv_data":380483775907678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030683089,"port_rcv_data":1403123601365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.512","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:16.524","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:16.535","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.546","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.556","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.566","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.575","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.585","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.3876317,"monotonic":2349885.834645372,"hcas":{"mlx5_0":{"port_xmit_data":380080367131655,"port_rcv_data":380483775907678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030683089,"port_rcv_data":1403123601365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.491","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.503","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.515","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.526","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.536","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.546","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.555","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.565","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.3884482,"monotonic":2349886.835462174,"hcas":{"mlx5_0":{"port_xmit_data":380080367131655,"port_rcv_data":380483775907678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868030683089,"port_rcv_data":1403123601365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.488","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.500","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:18.511","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.522","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.532","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.541","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.551","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.560","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.3892741,"monotonic":2349887.836288049,"hcas":{"mlx5_0":{"port_xmit_data":380080367468973,"port_rcv_data":380483895240508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868031033705,"port_rcv_data":1403242947617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.496","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":133.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.508","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":131.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.521","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":140.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.533","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":141.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.544","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.553","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.563","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.572","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.3900833,"monotonic":2349888.837097089,"hcas":{"mlx5_0":{"port_xmit_data":380080482643395,"port_rcv_data":380484009641328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868146203378,"port_rcv_data":1403357314214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416776699990,"port_rcv_data":380150724451859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436655302288,"port_rcv_data":1038521683571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.494","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":263.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:20.506","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":312.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:20.519","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":271.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.531","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":257.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.542","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":150.47,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.554","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":176.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.564","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":148.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.573","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":166.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.39091,"monotonic":2349889.837923165,"hcas":{"mlx5_0":{"port_xmit_data":380080482643395,"port_rcv_data":380484009641328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868146203378,"port_rcv_data":1403357314214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416844283613,"port_rcv_data":380150725739746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436726035088,"port_rcv_data":1038523091017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.531","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":167.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.546","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":161.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.557","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":163.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.568","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":174.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.578","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":268.83,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.588","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":254.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.598","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":242.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.607","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":255.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.391693,"monotonic":2349890.838706109,"hcas":{"mlx5_0":{"port_xmit_data":380080618492741,"port_rcv_data":380484009787031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868246049766,"port_rcv_data":1403357424016,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379416962378860,"port_rcv_data":380150727941667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436840980632,"port_rcv_data":1038525213045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.476","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.489","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.500","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.510","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.03,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.519","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":186.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.529","gpu_util_pct":1.0,"memory_used_mib":83581.0,"power_w":169.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.538","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":201.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.547","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":185.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.392532,"monotonic":2349891.839545475,"hcas":{"mlx5_0":{"port_xmit_data":380080618493172,"port_rcv_data":380484009787124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868246050197,"port_rcv_data":1403357424109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417091256491,"port_rcv_data":380150730158337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1436964141096,"port_rcv_data":1038527422368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.535","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.548","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.570","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.579","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.589","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.598","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.607","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.3933122,"monotonic":2349892.840325848,"hcas":{"mlx5_0":{"port_xmit_data":380080618495573,"port_rcv_data":380484009787728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868246052990,"port_rcv_data":1403357424760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417223285669,"port_rcv_data":380150732460676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437103187569,"port_rcv_data":1038529848366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.521","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.533","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.542","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.559","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.567","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.576","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.584","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.394133,"monotonic":2349893.841145496,"hcas":{"mlx5_0":{"port_xmit_data":380080618497051,"port_rcv_data":380484009788162,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868246054990,"port_rcv_data":1403357425256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417312034944,"port_rcv_data":380150734027664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437190635563,"port_rcv_data":1038531373272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.495","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.510","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.521","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.532","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.542","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.552","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.561","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.570","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.3949513,"monotonic":2349894.841964652,"hcas":{"mlx5_0":{"port_xmit_data":380080618497421,"port_rcv_data":380484009788271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868246055490,"port_rcv_data":1403357425380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417338145204,"port_rcv_data":380150734478931,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437216745676,"port_rcv_data":1038531833436,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.517","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.529","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.540","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.551","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.562","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.573","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.592","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.395738,"monotonic":2349895.842751277,"hcas":{"mlx5_0":{"port_xmit_data":380080618840922,"port_rcv_data":380484129126788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868246406613,"port_rcv_data":1403476771210,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417338145204,"port_rcv_data":380150734478931,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437216745676,"port_rcv_data":1038531833436,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.542","gpu_util_pct":50.0,"memory_used_mib":72407.0,"power_w":221.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.553","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":185.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.563","gpu_util_pct":50.0,"memory_used_mib":72407.0,"power_w":228.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.572","gpu_util_pct":25.0,"memory_used_mib":72407.0,"power_w":203.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.582","gpu_util_pct":93.0,"memory_used_mib":83581.0,"power_w":82.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.591","gpu_util_pct":98.0,"memory_used_mib":83581.0,"power_w":94.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.600","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":99.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.608","gpu_util_pct":98.0,"memory_used_mib":83581.0,"power_w":99.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.3965192,"monotonic":2349896.843532592,"hcas":{"mlx5_0":{"port_xmit_data":380080734013597,"port_rcv_data":380484243316301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868361578021,"port_rcv_data":1403590939271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417338145547,"port_rcv_data":380150734479326,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437216745676,"port_rcv_data":1038531833436,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.502","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":301.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.513","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":295.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.521","gpu_util_pct":94.0,"memory_used_mib":72407.0,"power_w":297.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:28.530","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":295.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.538","gpu_util_pct":55.0,"memory_used_mib":83581.0,"power_w":253.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.547","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":211.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.555","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":230.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.564","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":236.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.3970366,"monotonic":2349897.844049693,"hcas":{"mlx5_0":{"port_xmit_data":380080734013597,"port_rcv_data":380484243316301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868361578021,"port_rcv_data":1403590939271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417458905457,"port_rcv_data":380150736765505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437338181116,"port_rcv_data":1038534131621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.535","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.562","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.38,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.581","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":282.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.591","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":281.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.601","gpu_util_pct":50.0,"memory_used_mib":83581.0,"power_w":263.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.610","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":272.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.3978245,"monotonic":2349898.844837842,"hcas":{"mlx5_0":{"port_xmit_data":380080734013597,"port_rcv_data":380484243316301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868361578021,"port_rcv_data":1403590939271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417594552257,"port_rcv_data":380150739298391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437471861116,"port_rcv_data":1038536565384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.482","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.497","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.508","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.529","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":111.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.540","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":95.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.549","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.558","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":106.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.398661,"monotonic":2349899.845674365,"hcas":{"mlx5_0":{"port_xmit_data":380080734013797,"port_rcv_data":380484243316449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868361578021,"port_rcv_data":1403590939271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417621077057,"port_rcv_data":380150739777666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437499676876,"port_rcv_data":1038537088432,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.518","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.532","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.554","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.564","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.573","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.592","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.3994737,"monotonic":2349900.846487171,"hcas":{"mlx5_0":{"port_xmit_data":380080869862878,"port_rcv_data":380484243459908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868461424409,"port_rcv_data":1403591046019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417621081673,"port_rcv_data":380150739778906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437499681620,"port_rcv_data":1038537089672,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.505","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.517","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:32.529","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:32.539","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.550","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.560","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.569","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.578","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.4002957,"monotonic":2349901.847308866,"hcas":{"mlx5_0":{"port_xmit_data":380080869863463,"port_rcv_data":380484243460063,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868461425125,"port_rcv_data":1403591046190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417655895266,"port_rcv_data":380150740405447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437534495051,"port_rcv_data":1038537705208,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.527","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.542","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.553","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.563","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.573","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.593","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.602","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.401108,"monotonic":2349902.848121138,"hcas":{"mlx5_0":{"port_xmit_data":380080870185665,"port_rcv_data":380484362776505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868461768493,"port_rcv_data":1403710383608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417775322546,"port_rcv_data":380150742453028,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437653289959,"port_rcv_data":1038539750712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.530","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":185.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.544","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":193.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.557","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":206.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.567","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":209.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.578","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.587","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.597","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.606","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.4018943,"monotonic":2349903.848907691,"hcas":{"mlx5_0":{"port_xmit_data":380080985335417,"port_rcv_data":380484480453274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868576901917,"port_rcv_data":1403828001268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417899176081,"port_rcv_data":380150744822637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437777774744,"port_rcv_data":1038542066954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.526","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":289.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.542","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":261.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:35.554","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":280.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:35.565","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":286.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.576","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":189.98,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.586","gpu_util_pct":88.0,"memory_used_mib":83581.0,"power_w":187.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.596","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":187.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.605","gpu_util_pct":38.0,"memory_used_mib":83581.0,"power_w":182.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.4026923,"monotonic":2349904.849705356,"hcas":{"mlx5_0":{"port_xmit_data":380080985335718,"port_rcv_data":380484480453351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868576902201,"port_rcv_data":1403828001346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379417970323365,"port_rcv_data":380150746290315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437848921864,"port_rcv_data":1038543494101,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.542","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":157.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.557","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":164.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.568","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":129.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.579","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":151.65,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.590","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":234.74,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.600","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":254.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.609","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":275.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.619","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":254.36,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.4034731,"monotonic":2349905.850486521,"hcas":{"mlx5_0":{"port_xmit_data":380080985335718,"port_rcv_data":380484480453351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868576902201,"port_rcv_data":1403828001346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418094106368,"port_rcv_data":380150748576941,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1437976253159,"port_rcv_data":1038545710214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.551","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.566","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:37.577","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.587","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.597","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":163.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.606","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":151.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.616","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":125.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.624","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":146.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.4040322,"monotonic":2349906.851045337,"hcas":{"mlx5_0":{"port_xmit_data":380080985335718,"port_rcv_data":380484480453351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868576902201,"port_rcv_data":1403828001346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418182522368,"port_rcv_data":380150750163025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438061120264,"port_rcv_data":1038547329561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.487","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.502","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.513","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.523","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.532","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":103.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.542","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":111.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.551","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":99.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.560","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":99.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.4048774,"monotonic":2349907.851890867,"hcas":{"mlx5_0":{"port_xmit_data":380080985335718,"port_rcv_data":380484480453351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868576902201,"port_rcv_data":1403828001346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418182522368,"port_rcv_data":380150750163025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438061120264,"port_rcv_data":1038547329561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.521","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.532","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.554","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.564","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.574","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.584","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.593","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.4056878,"monotonic":2349908.85270158,"hcas":{"mlx5_0":{"port_xmit_data":380080985335918,"port_rcv_data":380484480453499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868576902201,"port_rcv_data":1403828001346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418182522368,"port_rcv_data":380150750163025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438061120264,"port_rcv_data":1038547329561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.503","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.515","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.526","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.537","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.98,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.547","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.556","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.565","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.574","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.406517,"monotonic":2349909.853530432,"hcas":{"mlx5_0":{"port_xmit_data":380081125716686,"port_rcv_data":380484599942215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868680090495,"port_rcv_data":1403947457449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418182526984,"port_rcv_data":380150750164265,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438061125008,"port_rcv_data":1038547330801,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.505","gpu_util_pct":29.0,"memory_used_mib":72407.0,"power_w":107.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.518","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":114.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.529","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":131.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.542","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":124.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.552","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.562","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.571","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.579","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.4073615,"monotonic":2349910.854374009,"hcas":{"mlx5_0":{"port_xmit_data":380081240857237,"port_rcv_data":380484712973295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868795258366,"port_rcv_data":1404060486656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418242786144,"port_rcv_data":380150751390718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438116956891,"port_rcv_data":1038548322289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.528","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":317.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:42.543","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":283.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:42.554","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":317.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.564","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":271.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.575","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":177.88,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.586","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":166.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.596","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":193.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.605","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":193.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.4081738,"monotonic":2349911.855186819,"hcas":{"mlx5_0":{"port_xmit_data":380081240859723,"port_rcv_data":380484712973930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868795261228,"port_rcv_data":1404060487338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418370152478,"port_rcv_data":380150753786234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438249490591,"port_rcv_data":1038550963505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.513","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":222.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.527","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":238.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.538","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":199.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.549","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":199.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.559","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":246.69,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.569","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":243.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.580","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":246.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.591","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":253.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.408667,"monotonic":2349912.855680388,"hcas":{"mlx5_0":{"port_xmit_data":380081240861332,"port_rcv_data":380484712974380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868795263375,"port_rcv_data":1404060487849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418487560924,"port_rcv_data":380150755930908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438366157124,"port_rcv_data":1038553096646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.511","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.524","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.534","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.545","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.99,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.555","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":228.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.564","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":211.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.574","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":226.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":226.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.4090168,"monotonic":2349913.856030068,"hcas":{"mlx5_0":{"port_xmit_data":380081240861597,"port_rcv_data":380484712974593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868795263375,"port_rcv_data":1404060487849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418611343324,"port_rcv_data":380150758141859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438489609524,"port_rcv_data":1038555296468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:45.547","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:45.560","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:45.570","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.580","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.590","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":87.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.599","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":106.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.608","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":103.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.617","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":103.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.4098072,"monotonic":2349914.856820601,"hcas":{"mlx5_0":{"port_xmit_data":380081355351186,"port_rcv_data":380484713097176,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868856199039,"port_rcv_data":1404060555336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418717446524,"port_rcv_data":380150760030333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438596041808,"port_rcv_data":1038557207695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.511","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.525","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.536","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.546","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.556","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.565","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.574","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.410644,"monotonic":2349915.857656101,"hcas":{"mlx5_0":{"port_xmit_data":380081375662748,"port_rcv_data":380484713119512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868894362159,"port_rcv_data":1404060594582,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418778717662,"port_rcv_data":380150761118712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438657312800,"port_rcv_data":1038558290603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.535","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.562","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.573","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.19,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.593","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.603","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.612","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.411445,"monotonic":2349916.858457803,"hcas":{"mlx5_0":{"port_xmit_data":380081375664718,"port_rcv_data":380484713120023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868894364521,"port_rcv_data":1404060595140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379418907818004,"port_rcv_data":380150763385509,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438786384753,"port_rcv_data":1038560554316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.534","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.545","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.555","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.565","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.575","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.585","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.595","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.412259,"monotonic":2349917.859272226,"hcas":{"mlx5_0":{"port_xmit_data":380081375999474,"port_rcv_data":380484832449466,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1868894707984,"port_rcv_data":1404179932738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419003210015,"port_rcv_data":380150765022728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438881804047,"port_rcv_data":1038562210584,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.515","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":184.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.531","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":192.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.542","gpu_util_pct":26.0,"memory_used_mib":72407.0,"power_w":172.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.553","gpu_util_pct":51.0,"memory_used_mib":72407.0,"power_w":158.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.563","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.573","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.593","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.4130774,"monotonic":2349918.860090199,"hcas":{"mlx5_0":{"port_xmit_data":380081491157558,"port_rcv_data":380484950708899,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869009866348,"port_rcv_data":1404298183953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419029112981,"port_rcv_data":380150765507244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1438907706752,"port_rcv_data":1038562694509,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.537","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":288.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.552","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":272.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.564","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":288.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.577","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":295.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.588","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":156.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.599","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":187.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.609","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":162.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.619","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":172.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.4138649,"monotonic":2349919.860878752,"hcas":{"mlx5_0":{"port_xmit_data":380081491485132,"port_rcv_data":380485070032792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869010193093,"port_rcv_data":1404417506675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419126370581,"port_rcv_data":380150767401685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439003208752,"port_rcv_data":1038564548139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.516","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":277.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.531","gpu_util_pct":59.0,"memory_used_mib":72407.0,"power_w":271.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":260.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.553","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":231.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.563","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":280.68,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.573","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":303.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.583","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":334.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.594","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":313.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.4146817,"monotonic":2349920.861695126,"hcas":{"mlx5_0":{"port_xmit_data":380081606647321,"port_rcv_data":380485186867728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869125368180,"port_rcv_data":1404534338094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419250153662,"port_rcv_data":380150769862509,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439128746752,"port_rcv_data":1038567043065,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:52.535","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":265.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:52.551","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":298.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.563","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":294.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.574","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":278.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.584","gpu_util_pct":44.0,"memory_used_mib":83581.0,"power_w":309.86,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.594","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":320.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.604","gpu_util_pct":24.0,"memory_used_mib":83581.0,"power_w":270.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.614","gpu_util_pct":36.0,"memory_used_mib":83581.0,"power_w":294.94,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.4154778,"monotonic":2349921.862491498,"hcas":{"mlx5_0":{"port_xmit_data":380081606647321,"port_rcv_data":380485186867728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869125368180,"port_rcv_data":1404534338094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419373936197,"port_rcv_data":380150772182727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439249865392,"port_rcv_data":1038569253133,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.558","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":115.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.573","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.584","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":84.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":104.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.605","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":261.23,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.615","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":288.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.625","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":262.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.634","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":284.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.4162502,"monotonic":2349922.863263824,"hcas":{"mlx5_0":{"port_xmit_data":380081606647321,"port_rcv_data":380485186867728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869125368180,"port_rcv_data":1404534338094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419502735180,"port_rcv_data":380150774414471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439382637232,"port_rcv_data":1038571672869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.535","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.561","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.582","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":156.73,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.591","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":130.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":118.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.609","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":125.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.4170516,"monotonic":2349923.864065451,"hcas":{"mlx5_0":{"port_xmit_data":380081606647521,"port_rcv_data":380485186867876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869125368180,"port_rcv_data":1404534338094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419594976540,"port_rcv_data":380150776133530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439473569152,"port_rcv_data":1038573349814,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.534","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.561","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.31,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":111.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.593","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":106.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.603","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.613","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":101.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.4178526,"monotonic":2349924.864866176,"hcas":{"mlx5_0":{"port_xmit_data":380081746685320,"port_rcv_data":380485187016848,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869228206088,"port_rcv_data":1404534448516,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419594981156,"port_rcv_data":380150776134770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439473573896,"port_rcv_data":1038573351054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:56.521","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.534","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.545","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.557","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.567","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.577","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.587","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.597","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.4186819,"monotonic":2349925.865695142,"hcas":{"mlx5_0":{"port_xmit_data":380081746686705,"port_rcv_data":380485187017205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869228207734,"port_rcv_data":1404534448904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419698182647,"port_rcv_data":380150777905071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439574304006,"port_rcv_data":1038575014793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.555","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.570","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.581","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.591","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.600","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.609","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.618","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.628","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.4195113,"monotonic":2349926.866524043,"hcas":{"mlx5_0":{"port_xmit_data":380081867560484,"port_rcv_data":380485306473330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869349088542,"port_rcv_data":1404653911129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419818856557,"port_rcv_data":380150780101111,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439697448453,"port_rcv_data":1038577295215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:58.541","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":128.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.557","gpu_util_pct":80.0,"memory_used_mib":72407.0,"power_w":138.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.568","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":124.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.579","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":111.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.589","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.598","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.607","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.616","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.4203317,"monotonic":2349927.867344549,"hcas":{"mlx5_0":{"port_xmit_data":380081982713715,"port_rcv_data":380485424817583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869464244438,"port_rcv_data":1404772224457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379419926681985,"port_rcv_data":380150782048283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439804334908,"port_rcv_data":1038579184494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.565","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":281.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.580","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":297.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.592","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":279.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.604","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":274.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.614","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":183.99,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.626","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":172.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.637","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":162.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.647","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":175.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.4210057,"monotonic":2349928.868018822,"hcas":{"mlx5_0":{"port_xmit_data":380081982715901,"port_rcv_data":380485424818141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869464247015,"port_rcv_data":1404772225062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420063116492,"port_rcv_data":380150784681865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1439938724748,"port_rcv_data":1038581857268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.507","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":265.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.523","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":246.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.534","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":232.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.544","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":240.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.555","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":309.17,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.565","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":312.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.575","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":318.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.587","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":323.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.4218407,"monotonic":2349929.868853438,"hcas":{"mlx5_0":{"port_xmit_data":380081982717656,"port_rcv_data":380485424818606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869464249162,"port_rcv_data":1404772225573,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420181617541,"port_rcv_data":380150786886978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440063334904,"port_rcv_data":1038584132420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:01.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":90.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.573","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":85.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.584","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":89.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":102.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.604","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":221.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.614","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":200.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.623","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":211.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.632","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":201.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.422628,"monotonic":2349930.869640701,"hcas":{"mlx5_0":{"port_xmit_data":380081982717810,"port_rcv_data":380485424818668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869464249446,"port_rcv_data":1404772225651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420300100878,"port_rcv_data":380150789065565,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440178690040,"port_rcv_data":1038586186836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.574","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.585","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.605","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":109.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.614","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":120.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.623","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":108.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.632","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":104.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.4234116,"monotonic":2349931.870425013,"hcas":{"mlx5_0":{"port_xmit_data":380081982717810,"port_rcv_data":380485424818668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869464249446,"port_rcv_data":1404772225651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420402923838,"port_rcv_data":380150790846329,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440275947640,"port_rcv_data":1038587913660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.558","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.569","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.579","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.588","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.598","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.607","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.615","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.4242134,"monotonic":2349932.871226978,"hcas":{"mlx5_0":{"port_xmit_data":380081982717810,"port_rcv_data":380485424818668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869464249446,"port_rcv_data":1404772225651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420450408078,"port_rcv_data":380150791761867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440328997240,"port_rcv_data":1038588862973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.506","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.521","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.532","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.553","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.562","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.572","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.581","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.4250464,"monotonic":2349933.872060079,"hcas":{"mlx5_0":{"port_xmit_data":380081983093856,"port_rcv_data":380485544191166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869464600847,"port_rcv_data":1404891573788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420450408078,"port_rcv_data":380150791761867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440328997240,"port_rcv_data":1038588862973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.527","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":161.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.539","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":154.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.552","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":158.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.564","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":165.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.577","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.586","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.595","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.604","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.4258583,"monotonic":2349934.872872128,"hcas":{"mlx5_0":{"port_xmit_data":380082238295766,"port_rcv_data":380485661799623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869682602457,"port_rcv_data":1405009131093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420450412694,"port_rcv_data":380150791763107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440329001984,"port_rcv_data":1038588864213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:06.532","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":256.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.544","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":288.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.557","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":262.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.568","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":274.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.580","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":185.16,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.591","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":193.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.602","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":187.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.612","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":199.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.4266694,"monotonic":2349935.873681868,"hcas":{"mlx5_0":{"port_xmit_data":380082238296566,"port_rcv_data":380485661799824,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869682603388,"port_rcv_data":1405009131310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420521628891,"port_rcv_data":380150793243359,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440400217790,"port_rcv_data":1038590317323,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":140.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.587","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":150.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.598","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":161.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.608","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":161.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.618","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":233.27,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.629","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":237.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.639","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":233.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.649","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":245.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.4274375,"monotonic":2349936.874450346,"hcas":{"mlx5_0":{"port_xmit_data":380082238298906,"port_rcv_data":380485661800444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869682606250,"port_rcv_data":1405009131992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420638986764,"port_rcv_data":380150795408375,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440517574670,"port_rcv_data":1038592476907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.585","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.596","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.605","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.615","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":185.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.624","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":187.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.633","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":159.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.642","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":176.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.4282331,"monotonic":2349937.87524644,"hcas":{"mlx5_0":{"port_xmit_data":380082238300446,"port_rcv_data":380485661800863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869682608181,"port_rcv_data":1405009132457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420769006186,"port_rcv_data":380150797643258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440647744100,"port_rcv_data":1038594685331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.585","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.597","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.607","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.617","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.627","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.637","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.646","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.4290516,"monotonic":2349938.876064178,"hcas":{"mlx5_0":{"port_xmit_data":380082238300446,"port_rcv_data":380485661800863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869682608181,"port_rcv_data":1405009132457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420905753866,"port_rcv_data":380150800193008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440782150100,"port_rcv_data":1038597142686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.555","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.570","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.580","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.590","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.599","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.608","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.618","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.627","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.4298556,"monotonic":2349939.876868822,"hcas":{"mlx5_0":{"port_xmit_data":380082238604269,"port_rcv_data":380485781101717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869682917691,"port_rcv_data":1405128438394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379420994169866,"port_rcv_data":380150801784347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440872757300,"port_rcv_data":1038598795404,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.535","gpu_util_pct":95.0,"memory_used_mib":72407.0,"power_w":113.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.550","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":118.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.562","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":110.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.572","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":109.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.584","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.594","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.603","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.612","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.43067,"monotonic":2349940.877683017,"hcas":{"mlx5_0":{"port_xmit_data":380082353793286,"port_rcv_data":380485897952656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869798108236,"port_rcv_data":1405245248884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421020694666,"port_rcv_data":380150802260460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440899282100,"port_rcv_data":1038599260280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.551","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":265.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.564","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":278.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.577","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":265.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.588","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":268.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.598","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":116.71,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.608","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":128.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.619","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":137.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.629","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":135.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.431468,"monotonic":2349941.878481089,"hcas":{"mlx5_0":{"port_xmit_data":380082474333732,"port_rcv_data":380485898080217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869918648092,"port_rcv_data":1405245377844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421056065959,"port_rcv_data":380150803070926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1440934653180,"port_rcv_data":1038599966538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:13.553","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":193.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.568","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":228.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.579","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":218.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.589","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":220.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.599","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":243.57,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.609","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":248.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.619","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":224.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.628","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":246.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.432282,"monotonic":2349942.879294487,"hcas":{"mlx5_0":{"port_xmit_data":380082474333732,"port_rcv_data":380485898080217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869918648092,"port_rcv_data":1405245377844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421190079052,"port_rcv_data":380150805453851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441059689820,"port_rcv_data":1038602188726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.555","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.570","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.581","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.592","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.602","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":267.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.611","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":224.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.620","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":227.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.629","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":228.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.4330876,"monotonic":2349943.880101047,"hcas":{"mlx5_0":{"port_xmit_data":380082474333947,"port_rcv_data":380485898080263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869918648307,"port_rcv_data":1405245377891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421321244979,"port_rcv_data":380150807993448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441199831971,"port_rcv_data":1038604886563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.569","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.584","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.606","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.615","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":111.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.625","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":110.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.634","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":102.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.644","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":101.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.43388,"monotonic":2349944.880892352,"hcas":{"mlx5_0":{"port_xmit_data":380082474336133,"port_rcv_data":380485898080821,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869918650885,"port_rcv_data":1405245378495,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421437290080,"port_rcv_data":380150810030458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441315876553,"port_rcv_data":1038606921403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:16.536","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.561","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.582","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.592","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.609","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.4347174,"monotonic":2349945.881730451,"hcas":{"mlx5_0":{"port_xmit_data":380082474337938,"port_rcv_data":380485898081453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869918652816,"port_rcv_data":1405245378960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421535306968,"port_rcv_data":380150811773092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441420478399,"port_rcv_data":1038608740096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.519","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.532","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.553","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.41,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.563","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.572","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.581","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.591","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.435571,"monotonic":2349946.88258409,"hcas":{"mlx5_0":{"port_xmit_data":380082474338677,"port_rcv_data":380485898081670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1869918653816,"port_rcv_data":1405245379208,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421588770820,"port_rcv_data":380150812715632,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441467356528,"port_rcv_data":1038609609754,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.558","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.570","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.581","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.592","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.602","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.612","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.621","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.631","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.4363837,"monotonic":2349947.883397466,"hcas":{"mlx5_0":{"port_xmit_data":380082595219903,"port_rcv_data":380486017546840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870039535922,"port_rcv_data":1405364847295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421588775500,"port_rcv_data":380150812716872,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441467361208,"port_rcv_data":1038609610994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.536","gpu_util_pct":59.0,"memory_used_mib":72407.0,"power_w":272.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.548","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":222.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":251.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.571","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":249.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.581","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.590","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.597","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.605","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.4372303,"monotonic":2349948.884243227,"hcas":{"mlx5_0":{"port_xmit_data":380082710427023,"port_rcv_data":380486136981603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870154744379,"port_rcv_data":1405484281077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421718470275,"port_rcv_data":380150815183255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441596891457,"port_rcv_data":1038612150725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.579","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":303.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.590","gpu_util_pct":95.0,"memory_used_mib":72407.0,"power_w":339.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.599","gpu_util_pct":95.0,"memory_used_mib":72407.0,"power_w":295.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.609","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":291.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.618","gpu_util_pct":12.0,"memory_used_mib":83581.0,"power_w":221.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.627","gpu_util_pct":67.0,"memory_used_mib":83581.0,"power_w":249.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.637","gpu_util_pct":30.0,"memory_used_mib":83581.0,"power_w":233.78,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.645","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":236.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.4380438,"monotonic":2349949.88505672,"hcas":{"mlx5_0":{"port_xmit_data":380082825407078,"port_rcv_data":380486177367977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870269736971,"port_rcv_data":1405524658114,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421855818132,"port_rcv_data":380150817852308,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441734402343,"port_rcv_data":1038614779432,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:21.561","gpu_util_pct":20.0,"memory_used_mib":72407.0,"power_w":171.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.577","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":168.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.588","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":172.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.599","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":208.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.609","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":288.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.621","gpu_util_pct":58.0,"memory_used_mib":83581.0,"power_w":285.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.630","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":292.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.640","gpu_util_pct":42.0,"memory_used_mib":83581.0,"power_w":301.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.438849,"monotonic":2349950.885861842,"hcas":{"mlx5_0":{"port_xmit_data":380082825408225,"port_rcv_data":380486177368705,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870269737540,"port_rcv_data":1405524658269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379421971174172,"port_rcv_data":380150819956674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441849757356,"port_rcv_data":1038616916107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.544","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.571","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.582","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.81,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.592","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":198.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.602","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":188.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.611","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":169.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.621","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":183.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.43968,"monotonic":2349951.88669284,"hcas":{"mlx5_0":{"port_xmit_data":380082872355359,"port_rcv_data":380486177426748,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870316684084,"port_rcv_data":1405524715433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422100484372,"port_rcv_data":380150822195479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1441981578196,"port_rcv_data":1038619259645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.547","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.562","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.573","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.582","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.591","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.600","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.609","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.617","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.4405184,"monotonic":2349952.887530992,"hcas":{"mlx5_0":{"port_xmit_data":380082872355359,"port_rcv_data":380486177426748,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870316684084,"port_rcv_data":1405524715433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422227585252,"port_rcv_data":380150824526911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442104581796,"port_rcv_data":1038621419213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.582","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.597","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.608","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.618","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.628","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.638","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.647","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.656","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.441036,"monotonic":2349953.888048848,"hcas":{"mlx5_0":{"port_xmit_data":380082872356437,"port_rcv_data":380486177426981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870316685162,"port_rcv_data":1405524715666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422349916487,"port_rcv_data":380150826697919,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442228499591,"port_rcv_data":1038623663799,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.546","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.561","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.582","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.591","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.610","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.619","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.4416952,"monotonic":2349954.888707782,"hcas":{"mlx5_0":{"port_xmit_data":380082872357083,"port_rcv_data":380486177427120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870316685808,"port_rcv_data":1405524715805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422439230108,"port_rcv_data":380150828284111,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442317813164,"port_rcv_data":1038625257928,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.574","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.585","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.604","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.614","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.623","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.632","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.4425015,"monotonic":2349955.889514892,"hcas":{"mlx5_0":{"port_xmit_data":380082872703523,"port_rcv_data":380486296767937,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870317043276,"port_rcv_data":1405644065846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422543463531,"port_rcv_data":380150830027396,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442422045415,"port_rcv_data":1038627000236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.561","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":230.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.572","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":238.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.580","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":242.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.589","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":153.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.597","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.604","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.611","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.618","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.4433277,"monotonic":2349956.89034091,"hcas":{"mlx5_0":{"port_xmit_data":380082987818775,"port_rcv_data":380486409400772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870432183803,"port_rcv_data":1405756716419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422543671075,"port_rcv_data":380150830028193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442422252648,"port_rcv_data":1038627000655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.542","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":276.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.552","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":288.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.562","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":271.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.571","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":277.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.579","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":190.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.587","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":192.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.595","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":186.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.603","gpu_util_pct":72.0,"memory_used_mib":83581.0,"power_w":185.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.44418,"monotonic":2349957.891193593,"hcas":{"mlx5_0":{"port_xmit_data":380082987818775,"port_rcv_data":380486409400772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870432183803,"port_rcv_data":1405756716419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422658611875,"port_rcv_data":380150832392718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442537193448,"port_rcv_data":1038629350734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.562","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":86.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.573","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":109.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.581","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":110.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.589","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":130.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.597","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":301.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.606","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":275.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.615","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":315.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.623","gpu_util_pct":95.0,"memory_used_mib":83581.0,"power_w":321.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.4450264,"monotonic":2349958.892039669,"hcas":{"mlx5_0":{"port_xmit_data":380083103871981,"port_rcv_data":380486409524343,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548236187,"port_rcv_data":1405756840466,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422808923968,"port_rcv_data":380150835105932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442687505328,"port_rcv_data":1038632058429,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.580","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.590","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.598","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.607","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.614","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":136.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.622","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":105.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.629","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.635","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":107.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.445834,"monotonic":2349959.892847473,"hcas":{"mlx5_0":{"port_xmit_data":380083103874122,"port_rcv_data":380486409524917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548238687,"port_rcv_data":1405756841086,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379422963443941,"port_rcv_data":380150837810419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442842024798,"port_rcv_data":1038634759567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.553","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.561","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.569","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.577","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.584","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.591","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.598","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.4466858,"monotonic":2349960.893699362,"hcas":{"mlx5_0":{"port_xmit_data":380083103876377,"port_rcv_data":380486409525506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241480,"port_rcv_data":1405756841737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423083564435,"port_rcv_data":380150839875267,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442962144642,"port_rcv_data":1038636815797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.579","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.589","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.597","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.605","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.613","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.620","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.627","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.634","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.4475014,"monotonic":2349961.894514696,"hcas":{"mlx5_0":{"port_xmit_data":380083103876661,"port_rcv_data":380486409525583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241911,"port_rcv_data":1405756841830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.598","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.620","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.640","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.650","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.659","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.668","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.4482784,"monotonic":2349962.89529162,"hcas":{"mlx5_0":{"port_xmit_data":380083103876661,"port_rcv_data":380486409525583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241911,"port_rcv_data":1405756841830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.598","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.621","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.641","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.651","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.669","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.449019,"monotonic":2349963.896032642,"hcas":{"mlx5_0":{"port_xmit_data":380083103876661,"port_rcv_data":380486409525583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241911,"port_rcv_data":1405756841830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.592","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.604","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.615","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.625","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.635","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.645","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.4498043,"monotonic":2349964.896818208,"hcas":{"mlx5_0":{"port_xmit_data":380083103876661,"port_rcv_data":380486409525583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241911,"port_rcv_data":1405756841830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.549","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.560","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.571","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.582","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.592","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.610","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.619","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.4506302,"monotonic":2349965.897644064,"hcas":{"mlx5_0":{"port_xmit_data":380083103876661,"port_rcv_data":380486409525583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241911,"port_rcv_data":1405756841830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.548","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.571","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.581","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.591","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.610","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.619","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.4514546,"monotonic":2349966.898468335,"hcas":{"mlx5_0":{"port_xmit_data":380083103876661,"port_rcv_data":380486409525583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241911,"port_rcv_data":1405756841830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.556","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.567","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.578","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.589","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.599","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.608","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.617","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.626","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.4522731,"monotonic":2349967.899286805,"hcas":{"mlx5_0":{"port_xmit_data":380083103876661,"port_rcv_data":380486409525583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548241911,"port_rcv_data":1405756841830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.531","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.543","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.554","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.564","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.574","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.583","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.592","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.4531236,"monotonic":2349968.900137346,"hcas":{"mlx5_0":{"port_xmit_data":380083104228944,"port_rcv_data":380486528874696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870548580347,"port_rcv_data":1405876176740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.554","gpu_util_pct":94.0,"memory_used_mib":72407.0,"power_w":91.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.565","gpu_util_pct":95.0,"memory_used_mib":72407.0,"power_w":98.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.576","gpu_util_pct":95.0,"memory_used_mib":72407.0,"power_w":90.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.586","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":89.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.596","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.603","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.610","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.616","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.4539537,"monotonic":2349969.900967701,"hcas":{"mlx5_0":{"port_xmit_data":380083219357694,"port_rcv_data":380486644493212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870663714423,"port_rcv_data":1405991774875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423100694920,"port_rcv_data":380150840173766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1442979274980,"port_rcv_data":1038637106779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.559","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":315.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.569","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":288.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.578","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":317.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.587","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":284.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.595","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":126.5,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.603","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":125.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.612","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":128.45,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.621","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":193.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.4547892,"monotonic":2349970.901802781,"hcas":{"mlx5_0":{"port_xmit_data":380083219357694,"port_rcv_data":380486644493212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870663714423,"port_rcv_data":1405991774875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423144903133,"port_rcv_data":380150841043658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443023482073,"port_rcv_data":1038638006879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.597","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":203.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.607","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":206.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.615","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":204.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.623","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":206.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.631","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":256.38,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.639","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":262.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.648","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":245.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.655","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":243.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.4555902,"monotonic":2349971.902603866,"hcas":{"mlx5_0":{"port_xmit_data":380083338103396,"port_rcv_data":380486644620036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782459367,"port_rcv_data":1405991901324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423286373561,"port_rcv_data":380150843720135,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443164953260,"port_rcv_data":1038640748194,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.592","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.603","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.611","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.620","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.627","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":244.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.635","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":224.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.642","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":214.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.649","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":219.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.4563959,"monotonic":2349972.903409592,"hcas":{"mlx5_0":{"port_xmit_data":380083338104196,"port_rcv_data":380486644620237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782460298,"port_rcv_data":1405991901541,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423436058761,"port_rcv_data":380150846377780,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443314638282,"port_rcv_data":1038643440358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.582","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.590","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.598","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.605","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.612","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.619","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.626","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.4568017,"monotonic":2349973.903814174,"hcas":{"mlx5_0":{"port_xmit_data":380083338106752,"port_rcv_data":380486644620904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782463376,"port_rcv_data":1405991902269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423576625420,"port_rcv_data":380150848812381,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443455204275,"port_rcv_data":1038645887188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.559","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.573","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.585","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.605","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.614","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.624","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.634","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.4570346,"monotonic":2349974.904047836,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.579","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.591","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.602","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.614","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.8,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.624","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.634","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.643","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.653","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.4578404,"monotonic":2349975.90485371,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.602","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.613","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.624","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.645","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.655","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.664","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.4586222,"monotonic":2349976.905635913,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.604","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.616","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.627","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.11,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.637","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.647","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.656","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.665","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.4594054,"monotonic":2349977.906419065,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.608","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.619","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.651","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.661","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.670","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.679","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.4601822,"monotonic":2349978.907196227,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.620","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.651","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.669","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.678","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.4609594,"monotonic":2349979.907973254,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.604","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.616","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.627","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.637","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.647","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.656","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.665","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.4617467,"monotonic":2349980.908760456,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.608","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.619","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.630","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.651","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:52.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:52.669","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:52.678","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.4624102,"monotonic":2349981.909423715,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.601","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.613","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.624","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.646","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.656","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.665","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.4631941,"monotonic":2349982.910207724,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.604","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.616","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.627","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.638","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.649","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.658","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.668","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:54.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.4639752,"monotonic":2349983.910988782,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.611","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.623","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.656","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.666","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.675","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.684","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.4647508,"monotonic":2349984.91176443,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.603","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.615","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.626","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.637","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.648","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.658","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.667","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.676","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.4655378,"monotonic":2349985.912551576,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.583","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.607","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.618","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.628","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.638","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.647","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.657","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.466345,"monotonic":2349986.913358657,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.587","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.599","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.610","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.621","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.631","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.641","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.650","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.659","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.4671497,"monotonic":2349987.914163572,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.611","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.623","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.634","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.656","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.666","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.675","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.684","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.46793,"monotonic":2349988.914943814,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.550","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.561","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.572","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.583","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.592","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.601","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.611","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.619","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.4687736,"monotonic":2349989.915787415,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.608","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.619","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.630","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.650","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.669","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.4690108,"monotonic":2349990.916024583,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.613","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.624","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.657","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.666","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.675","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.684","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.4697883,"monotonic":2349991.916802162,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.613","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.625","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.637","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.648","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.658","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.667","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.686","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.4705648,"monotonic":2349992.917578691,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.568","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.580","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.591","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.602","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.612","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.622","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.631","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.641","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.4713862,"monotonic":2349993.918399816,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.567","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.579","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.590","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":76.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.601","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.611","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.621","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.631","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.640","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.472208,"monotonic":2349994.919221957,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.551","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.563","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.574","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.585","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.596","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.606","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.615","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.624","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.4730477,"monotonic":2349995.920061599,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.552","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.564","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.575","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.586","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.596","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.606","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.615","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:07.624","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.4738894,"monotonic":2349996.920903058,"hcas":{"mlx5_0":{"port_xmit_data":380083338108076,"port_rcv_data":380486644621276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870782465091,"port_rcv_data":1405991902688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.553","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.565","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.576","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.587","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.598","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.608","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.617","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.626","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.474726,"monotonic":2349997.921739793,"hcas":{"mlx5_0":{"port_xmit_data":380083380560583,"port_rcv_data":380486780892253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1870825930026,"port_rcv_data":1406128118848,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350273,"port_rcv_data":380150850328203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.575","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":242.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.585","gpu_util_pct":17.0,"memory_used_mib":72407.0,"power_w":205.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.597","gpu_util_pct":3.0,"memory_used_mib":72407.0,"power_w":245.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.607","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":224.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.616","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.625","gpu_util_pct":89.0,"memory_used_mib":83581.0,"power_w":82.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.635","gpu_util_pct":11.0,"memory_used_mib":83581.0,"power_w":96.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.643","gpu_util_pct":97.0,"memory_used_mib":83581.0,"power_w":94.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.4755483,"monotonic":2349998.922561947,"hcas":{"mlx5_0":{"port_xmit_data":380083568632234,"port_rcv_data":380486932538276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871013021598,"port_rcv_data":1406279850188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423664350876,"port_rcv_data":380150850328858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443542928672,"port_rcv_data":1038647419301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.591","gpu_util_pct":98.0,"memory_used_mib":72407.0,"power_w":294.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.601","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":315.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.610","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":292.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.619","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":304.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.629","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":244.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.640","gpu_util_pct":43.0,"memory_used_mib":83581.0,"power_w":203.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.649","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":204.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.660","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":222.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.4763694,"monotonic":2349999.923382599,"hcas":{"mlx5_0":{"port_xmit_data":380083568632234,"port_rcv_data":380486932538276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871013021598,"port_rcv_data":1406279850188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423779291787,"port_rcv_data":380150852601448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443657869472,"port_rcv_data":1038649668289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.622","gpu_util_pct":32.0,"memory_used_mib":72407.0,"power_w":197.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.635","gpu_util_pct":5.0,"memory_used_mib":72407.0,"power_w":188.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":194.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.657","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":169.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.666","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":335.53,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.677","gpu_util_pct":60.0,"memory_used_mib":83581.0,"power_w":307.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.687","gpu_util_pct":85.0,"memory_used_mib":83581.0,"power_w":311.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.697","gpu_util_pct":75.0,"memory_used_mib":83581.0,"power_w":334.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.477149,"monotonic":2350000.924162164,"hcas":{"mlx5_0":{"port_xmit_data":380083623657487,"port_rcv_data":380486932601920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871068045438,"port_rcv_data":1406279912870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379423915688724,"port_rcv_data":380150855106886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443792464615,"port_rcv_data":1038652077593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.600","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.615","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.626","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.636","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.99,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.645","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":216.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":188.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":191.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":196.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.4779665,"monotonic":2350001.924979218,"hcas":{"mlx5_0":{"port_xmit_data":380083623657487,"port_rcv_data":380486932601920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871068045438,"port_rcv_data":1406279912870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424035703604,"port_rcv_data":380150857355874,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1443914280552,"port_rcv_data":1038654406096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.598","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.613","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.624","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.634","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.644","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.653","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.672","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.478774,"monotonic":2350002.925787076,"hcas":{"mlx5_0":{"port_xmit_data":380083623657702,"port_rcv_data":380486932601966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871068045654,"port_rcv_data":1406279912916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424154512485,"port_rcv_data":380150859485217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444033089417,"port_rcv_data":1038656551684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.582","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.597","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.607","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.617","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.627","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.636","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.645","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.4796007,"monotonic":2350003.926614032,"hcas":{"mlx5_0":{"port_xmit_data":380083623659280,"port_rcv_data":380486932602323,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871068047378,"port_rcv_data":1406279913288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424268692642,"port_rcv_data":380150861503224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444147269331,"port_rcv_data":1038658572496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.599","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.614","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.624","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.634","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.644","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.672","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.4802852,"monotonic":2350004.927298393,"hcas":{"mlx5_0":{"port_xmit_data":380083623662167,"port_rcv_data":380486932603160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871068051162,"port_rcv_data":1406279914234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424353999484,"port_rcv_data":380150862934277,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444232575132,"port_rcv_data":1038659982547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.585","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.597","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.608","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.619","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.629","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.639","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.648","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.657","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.4811058,"monotonic":2350005.92811945,"hcas":{"mlx5_0":{"port_xmit_data":380083624019605,"port_rcv_data":380487051955893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871068395521,"port_rcv_data":1406399254539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424353999484,"port_rcv_data":380150862934277,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444232575132,"port_rcv_data":1038659982547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.584","gpu_util_pct":26.0,"memory_used_mib":72407.0,"power_w":197.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.596","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":207.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.607","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":199.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.618","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":200.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.629","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.638","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.648","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.657","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.4813719,"monotonic":2350006.928384707,"hcas":{"mlx5_0":{"port_xmit_data":380083739165851,"port_rcv_data":380487168062513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871183560026,"port_rcv_data":1406515366793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424353999697,"port_rcv_data":380150862934542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444232575132,"port_rcv_data":1038659982547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.583","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":276.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.594","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":302.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.605","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":275.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.616","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":277.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.626","gpu_util_pct":3.0,"memory_used_mib":83581.0,"power_w":210.43,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.635","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":208.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.644","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":199.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.653","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":198.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.482223,"monotonic":2350007.929235517,"hcas":{"mlx5_0":{"port_xmit_data":380083739165851,"port_rcv_data":380487168062513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871183560026,"port_rcv_data":1406515366793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424460098897,"port_rcv_data":380150865069760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444338674332,"port_rcv_data":1038662063685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.620","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":100.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":96.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.649","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.660","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.671","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":297.14,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.681","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":310.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.691","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":294.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.701","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":310.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.4830072,"monotonic":2350008.930020119,"hcas":{"mlx5_0":{"port_xmit_data":380083739165851,"port_rcv_data":380487168062513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871183560026,"port_rcv_data":1406515366793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424583519928,"port_rcv_data":380150867305114,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444459560652,"port_rcv_data":1038664192827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.588","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.601","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.612","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.623","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.633","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":89.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.642","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":123.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.652","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.661","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":107.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.4838376,"monotonic":2350009.930851215,"hcas":{"mlx5_0":{"port_xmit_data":380083739166051,"port_rcv_data":380487168062661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871183560026,"port_rcv_data":1406515366793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424636931208,"port_rcv_data":380150868267106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444515506332,"port_rcv_data":1038665250439,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.630","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.642","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.653","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.663","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.683","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.692","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:21.701","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.4846196,"monotonic":2350010.931633321,"hcas":{"mlx5_0":{"port_xmit_data":380083877108120,"port_rcv_data":380487168207160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871284902110,"port_rcv_data":1406515474580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424636936932,"port_rcv_data":380150868268470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444515511140,"port_rcv_data":1038665251679,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.584","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.596","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.607","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.617","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.627","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.636","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.645","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.4850667,"monotonic":2350011.932079518,"hcas":{"mlx5_0":{"port_xmit_data":380083877108120,"port_rcv_data":380487168207160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871284902110,"port_rcv_data":1406515474580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424671474444,"port_rcv_data":380150868889137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444550048652,"port_rcv_data":1038665881072,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.612","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.624","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.81,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.649","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:23.657","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:23.665","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:23.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.4858944,"monotonic":2350012.932907383,"hcas":{"mlx5_0":{"port_xmit_data":380083877428007,"port_rcv_data":380487287521384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871285232974,"port_rcv_data":1406634798421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424787243153,"port_rcv_data":380150870909810,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444665816695,"port_rcv_data":1038667914935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.591","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":150.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.606","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":128.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.618","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":145.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.631","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":144.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.641","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.651","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.669","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.4867122,"monotonic":2350013.9337255,"hcas":{"mlx5_0":{"port_xmit_data":380083992609381,"port_rcv_data":380487406929193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871400431258,"port_rcv_data":1406754222774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424901976086,"port_rcv_data":380150872968411,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444786298494,"port_rcv_data":1038670069838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.639","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":305.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.652","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":254.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.663","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":286.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.674","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":287.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:25.684","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":180.72,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:25.694","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":146.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:25.705","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":133.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:25.715","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":147.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.4874947,"monotonic":2350014.934507493,"hcas":{"mlx5_0":{"port_xmit_data":380084107607540,"port_rcv_data":380487455171956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871515439532,"port_rcv_data":1406802470632,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379424955233025,"port_rcv_data":380150874032457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444833805720,"port_rcv_data":1038671064892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.636","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":308.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.650","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":300.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.661","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":307.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.672","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":305.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.682","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":251.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.691","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":237.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.701","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":210.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.710","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":241.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.488277,"monotonic":2350015.93528996,"hcas":{"mlx5_0":{"port_xmit_data":380084107920518,"port_rcv_data":380487574479874,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871515770792,"port_rcv_data":1406921796661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379425089996008,"port_rcv_data":380150876548843,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1444958863480,"port_rcv_data":1038673356892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.635","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":234.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.651","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":208.27,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.663","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":227.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.674","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":188.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.685","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":337.88,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.695","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":335.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.704","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":333.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.713","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":340.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.4890618,"monotonic":2350016.936074417,"hcas":{"mlx5_0":{"port_xmit_data":380084223090291,"port_rcv_data":380487691197959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871630939966,"port_rcv_data":1407038486171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379425220481971,"port_rcv_data":380150879141175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445098005723,"port_rcv_data":1038676189688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.615","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":265.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.630","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":236.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.642","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":260.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.654","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":260.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.665","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":218.77,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.677","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":210.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.688","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":207.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.699","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":225.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.4898505,"monotonic":2350017.93686326,"hcas":{"mlx5_0":{"port_xmit_data":380084223414397,"port_rcv_data":380487810517996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871631244915,"port_rcv_data":1407157785625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379425335423114,"port_rcv_data":380150881321397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445213994520,"port_rcv_data":1038678508054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.632","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":340.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.647","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":356.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.659","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":340.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.670","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":315.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.681","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":319.3,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.691","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":304.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.702","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":279.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.712","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":310.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.49063,"monotonic":2350018.937642601,"hcas":{"mlx5_0":{"port_xmit_data":380084282900575,"port_rcv_data":380487918328426,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871802035370,"port_rcv_data":1407265715492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379425468047457,"port_rcv_data":380150883880796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445351634760,"port_rcv_data":1038681102816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.637","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":199.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.653","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":221.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.665","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":183.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.677","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":206.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.687","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":273.77,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.697","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":298.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.708","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":292.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.717","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":304.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.4913976,"monotonic":2350019.938411205,"hcas":{"mlx5_0":{"port_xmit_data":380084282901165,"port_rcv_data":380487918328964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871802035370,"port_rcv_data":1407265715492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379425591830460,"port_rcv_data":380150886414988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445470400920,"port_rcv_data":1038683609925,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":174.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.649","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":183.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.661","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":180.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.671","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":169.01,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.682","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":269.42,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.692","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":296.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.702","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":300.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.712","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":308.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.4921694,"monotonic":2350020.939183093,"hcas":{"mlx5_0":{"port_xmit_data":380084282901788,"port_rcv_data":380487918329535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871802035370,"port_rcv_data":1407265715492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379425728020275,"port_rcv_data":380150888699878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445577019228,"port_rcv_data":1038685769773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.647","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.658","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.668","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.678","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":152.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.687","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":136.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.696","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":168.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.705","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":171.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.4922934,"monotonic":2350021.939306159,"hcas":{"mlx5_0":{"port_xmit_data":380084282902150,"port_rcv_data":380487918329845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871802035370,"port_rcv_data":1407265715492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379425887346819,"port_rcv_data":380150891063767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445680869848,"port_rcv_data":1038688089146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.594","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.610","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.622","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.643","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.653","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:33.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.4930289,"monotonic":2350022.940041774,"hcas":{"mlx5_0":{"port_xmit_data":380084344688334,"port_rcv_data":380488037675634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871849060555,"port_rcv_data":1407385047576,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379426001143487,"port_rcv_data":380150892929688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445768425856,"port_rcv_data":1038689965235,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.612","gpu_util_pct":16.0,"memory_used_mib":72407.0,"power_w":245.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.627","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":232.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.639","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":240.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.649","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":233.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":84.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.672","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.681","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.688","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.4937336,"monotonic":2350023.940746328,"hcas":{"mlx5_0":{"port_xmit_data":380084459823069,"port_rcv_data":380488152224392,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1871964198793,"port_rcv_data":1407499565914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379426125021205,"port_rcv_data":380150895059565,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445864481232,"port_rcv_data":1038692069655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.622","gpu_util_pct":27.0,"memory_used_mib":72407.0,"power_w":295.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.634","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":283.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.644","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":292.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.654","gpu_util_pct":52.0,"memory_used_mib":72407.0,"power_w":294.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":271.74,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.672","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":254.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.680","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":216.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.689","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":223.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.4945507,"monotonic":2350024.941563807,"hcas":{"mlx5_0":{"port_xmit_data":380084580366107,"port_rcv_data":380488152354147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872084742056,"port_rcv_data":1407499695478,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379426262078976,"port_rcv_data":380150897503911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1445991872925,"port_rcv_data":1038694529489,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.642","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":83.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.655","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.666","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.677","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.688","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":322.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.698","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":295.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.708","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":298.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.717","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":290.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.4953382,"monotonic":2350025.942351545,"hcas":{"mlx5_0":{"port_xmit_data":380084708885970,"port_rcv_data":380488152492152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872179353909,"port_rcv_data":1407499799235,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379426412084914,"port_rcv_data":380150900075146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1446144191636,"port_rcv_data":1038697172022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.595","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.606","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.614","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.622","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.18,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.630","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":111.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.637","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":116.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.644","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":104.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.651","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.496185,"monotonic":2350026.943198318,"hcas":{"mlx5_0":{"port_xmit_data":380084709220336,"port_rcv_data":380488271822455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872179683794,"port_rcv_data":1407619124453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379426550382576,"port_rcv_data":380150902784747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1446307058624,"port_rcv_data":1038699799040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.643","gpu_util_pct":53.0,"memory_used_mib":72407.0,"power_w":250.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.652","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":267.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.660","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":245.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.666","gpu_util_pct":29.0,"memory_used_mib":72407.0,"power_w":244.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.672","gpu_util_pct":42.0,"memory_used_mib":83581.0,"power_w":109.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.680","gpu_util_pct":17.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.686","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":98.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.693","gpu_util_pct":64.0,"memory_used_mib":83581.0,"power_w":104.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.497005,"monotonic":2350027.944018537,"hcas":{"mlx5_0":{"port_xmit_data":380084824342461,"port_rcv_data":380488390354493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872294812240,"port_rcv_data":1407737633482,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379426694382795,"port_rcv_data":380150905850949,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1446487090730,"port_rcv_data":1038702922057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.598","gpu_util_pct":25.0,"memory_used_mib":72407.0,"power_w":288.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.609","gpu_util_pct":96.0,"memory_used_mib":72407.0,"power_w":298.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.617","gpu_util_pct":95.0,"memory_used_mib":72407.0,"power_w":293.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.625","gpu_util_pct":49.0,"memory_used_mib":72407.0,"power_w":304.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.632","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":221.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.642","gpu_util_pct":78.0,"memory_used_mib":83581.0,"power_w":222.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.649","gpu_util_pct":58.0,"memory_used_mib":83581.0,"power_w":262.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:39.656","gpu_util_pct":59.0,"memory_used_mib":83581.0,"power_w":250.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.4974263,"monotonic":2350028.944439584,"hcas":{"mlx5_0":{"port_xmit_data":380084824345301,"port_rcv_data":380488390355237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872294815749,"port_rcv_data":1407737634304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379426846100012,"port_rcv_data":380150908932397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1446661543049,"port_rcv_data":1038706013005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.615","gpu_util_pct":5.0,"memory_used_mib":72407.0,"power_w":81.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.627","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.636","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.645","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":93.74,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.654","gpu_util_pct":2.0,"memory_used_mib":83581.0,"power_w":282.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.661","gpu_util_pct":6.0,"memory_used_mib":83581.0,"power_w":279.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.668","gpu_util_pct":59.0,"memory_used_mib":83581.0,"power_w":275.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":277.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.498257,"monotonic":2350029.945270058,"hcas":{"mlx5_0":{"port_xmit_data":380084922936030,"port_rcv_data":380488564494923,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872393422095,"port_rcv_data":1407922457889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379427007997447,"port_rcv_data":380150911735929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1446823117943,"port_rcv_data":1038708768378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.599","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":255.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.610","gpu_util_pct":17.0,"memory_used_mib":72407.0,"power_w":245.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.619","gpu_util_pct":3.0,"memory_used_mib":72407.0,"power_w":276.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.628","gpu_util_pct":77.0,"memory_used_mib":72407.0,"power_w":259.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.637","gpu_util_pct":75.0,"memory_used_mib":83581.0,"power_w":94.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.645","gpu_util_pct":43.0,"memory_used_mib":83581.0,"power_w":84.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.652","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.660","gpu_util_pct":98.0,"memory_used_mib":83581.0,"power_w":84.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.4991057,"monotonic":2350030.946119284,"hcas":{"mlx5_0":{"port_xmit_data":380085075668520,"port_rcv_data":380488628314276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872510137296,"port_rcv_data":1407975545278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379427157415259,"port_rcv_data":380150914718631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1446978541235,"port_rcv_data":1038711893969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":290.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.644","gpu_util_pct":20.0,"memory_used_mib":72407.0,"power_w":287.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.652","gpu_util_pct":34.0,"memory_used_mib":72407.0,"power_w":280.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.661","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":269.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.668","gpu_util_pct":93.0,"memory_used_mib":83581.0,"power_w":253.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.676","gpu_util_pct":21.0,"memory_used_mib":83581.0,"power_w":271.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:42.683","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":289.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:42.691","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":273.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.4999154,"monotonic":2350031.946928722,"hcas":{"mlx5_0":{"port_xmit_data":380085075668605,"port_rcv_data":380488628314307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872510137365,"port_rcv_data":1407975545309,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379427322378510,"port_rcv_data":380150917862900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1447137068372,"port_rcv_data":1038714919522,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.599","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.610","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":85.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.618","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":88.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.626","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.633","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":310.28,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.641","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":290.03,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.648","gpu_util_pct":29.0,"memory_used_mib":83581.0,"power_w":286.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":307.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.5007722,"monotonic":2350032.947785579,"hcas":{"mlx5_0":{"port_xmit_data":380085320340078,"port_rcv_data":380488628573956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754806901,"port_rcv_data":1407975804547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379427473387410,"port_rcv_data":380150920548306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1447292570132,"port_rcv_data":1038717599472,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.617","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.627","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.643","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.650","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":84.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.658","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.664","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.671","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.5008688,"monotonic":2350033.947882433,"hcas":{"mlx5_0":{"port_xmit_data":380085320341509,"port_rcv_data":380488628574297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754808625,"port_rcv_data":1407975804919,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379427632798325,"port_rcv_data":380150923375274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1447451355835,"port_rcv_data":1038720405825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.599","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.617","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.625","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.633","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:45.641","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:45.648","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:45.655","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.501021,"monotonic":2350034.948033998,"hcas":{"mlx5_0":{"port_xmit_data":380085320344009,"port_rcv_data":380488628574917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754811857,"port_rcv_data":1407975805616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379427795310127,"port_rcv_data":380150926149262,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1447611689105,"port_rcv_data":1038723268148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.598","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.617","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.625","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.632","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.639","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.646","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.652","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.5018778,"monotonic":2350035.9488911,"hcas":{"mlx5_0":{"port_xmit_data":380085320346480,"port_rcv_data":380488628575553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754814866,"port_rcv_data":1407975806314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379427949862224,"port_rcv_data":380150928868629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1447772424951,"port_rcv_data":1038726038711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.599","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.617","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.625","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.632","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.640","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.647","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.653","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.5027323,"monotonic":2350036.949745729,"hcas":{"mlx5_0":{"port_xmit_data":380085320349104,"port_rcv_data":380488628576250,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754818159,"port_rcv_data":1407975807089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428112797372,"port_rcv_data":380150931711236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1447933920503,"port_rcv_data":1038728831392,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.599","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.617","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.625","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.633","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.73,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.640","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.647","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.653","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.503586,"monotonic":2350037.950599369,"hcas":{"mlx5_0":{"port_xmit_data":380085320351729,"port_rcv_data":380488628576948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754821452,"port_rcv_data":1407975807864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428273616471,"port_rcv_data":380150934460781,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448092711285,"port_rcv_data":1038731506661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.620","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.639","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.667","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.5044222,"monotonic":2350038.951435803,"hcas":{"mlx5_0":{"port_xmit_data":380085320353353,"port_rcv_data":380488628577397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754823452,"port_rcv_data":1407975808360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428375073580,"port_rcv_data":380150936240723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448196195443,"port_rcv_data":1038733368320,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.619","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.630","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.638","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.654","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.661","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:50.668","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:50.675","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.505254,"monotonic":2350039.952267768,"hcas":{"mlx5_0":{"port_xmit_data":380085320353693,"port_rcv_data":380488628577521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754823728,"port_rcv_data":1407975808484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428411268914,"port_rcv_data":380150936862893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448232390777,"port_rcv_data":1038733985731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.619","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.628","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.636","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:51.643","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.21,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:51.651","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:51.658","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.664","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.671","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.5060887,"monotonic":2350040.953102304,"hcas":{"mlx5_0":{"port_xmit_data":380085320354033,"port_rcv_data":380488628577645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824004,"port_rcv_data":1407975808608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428447464248,"port_rcv_data":380150937486071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448268586111,"port_rcv_data":1038734601298,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.638","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.647","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.655","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.663","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.670","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.684","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.690","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.5069115,"monotonic":2350041.95392476,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.649","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.661","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.671","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.681","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.691","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:53.700","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:53.709","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.719","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.5076985,"monotonic":2350042.954711655,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.657","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.668","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.679","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.689","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.699","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.708","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.717","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.726","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.5084774,"monotonic":2350043.955490626,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.658","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.670","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.681","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.691","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.94,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.701","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.710","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.719","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.728","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.5092576,"monotonic":2350044.956270976,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.652","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.664","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.675","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:56.686","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:56.696","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:56.705","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:56.714","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:56.724","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.510048,"monotonic":2350045.957061686,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.644","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.655","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.665","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.675","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.684","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.693","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.702","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.5108578,"monotonic":2350046.957871406,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.656","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.668","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.678","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.689","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.698","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:58.707","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.715","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.722","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.5116494,"monotonic":2350047.958662919,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.660","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.669","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.679","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.689","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.698","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.707","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.716","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.725","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.5124428,"monotonic":2350048.959456658,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.593","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.605","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.616","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:00.626","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:00.636","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:00.645","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:00.655","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:00.664","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.5130248,"monotonic":2350049.960038497,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.651","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.663","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.674","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.684","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.694","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.704","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.713","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.722","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.5138264,"monotonic":2350050.960839403,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.614","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.623","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.642","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:02.650","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:02.659","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.668","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.5146582,"monotonic":2350051.961672006,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.650","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.659","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.668","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.685","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.694","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.5154705,"monotonic":2350052.96248425,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.636","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.645","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.654","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.663","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.672","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.681","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.689","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.696","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.5156755,"monotonic":2350053.962689462,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.645","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.655","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.664","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.683","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.691","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.700","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.5164838,"monotonic":2350054.963497614,"hcas":{"mlx5_0":{"port_xmit_data":380085320354118,"port_rcv_data":380488628577676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872754824073,"port_rcv_data":1407975808639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.594","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.604","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.614","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.623","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:06.632","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:06.641","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.649","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.657","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.5173373,"monotonic":2350055.964351199,"hcas":{"mlx5_0":{"port_xmit_data":380085320703700,"port_rcv_data":380488747923220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872755170873,"port_rcv_data":1408095151757,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.636","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":157.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.647","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":159.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.657","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":154.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.666","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":153.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.681","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.687","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.694","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.518161,"monotonic":2350056.965174861,"hcas":{"mlx5_0":{"port_xmit_data":380085435825712,"port_rcv_data":380488865202960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872870299600,"port_rcv_data":1408212417039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428456513082,"port_rcv_data":380150937640745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448277634944,"port_rcv_data":1038734754531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.619","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":288.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.628","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":301.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.637","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":282.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.645","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":270.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.653","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":151.49,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.661","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":158.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.668","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":157.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.674","gpu_util_pct":82.0,"memory_used_mib":83581.0,"power_w":155.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.519009,"monotonic":2350057.966022882,"hcas":{"mlx5_0":{"port_xmit_data":380085435825712,"port_rcv_data":380488865202960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872870299600,"port_rcv_data":1408212417039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428556619695,"port_rcv_data":380150939576123,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448365615344,"port_rcv_data":1038736530816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":143.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.644","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":151.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.652","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":145.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.660","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":126.54,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.667","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":269.01,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.674","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":265.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.681","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":256.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.688","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":273.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259210.519846,"monotonic":2350058.966859189,"hcas":{"mlx5_0":{"port_xmit_data":380085435825712,"port_rcv_data":380488865202960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872870299600,"port_rcv_data":1408212417039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428691585883,"port_rcv_data":380150942049745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448513419824,"port_rcv_data":1038739188260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.636","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.654","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.662","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.669","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":177.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.676","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":185.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.683","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":158.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.690","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":164.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.5206778,"monotonic":2350059.967691635,"hcas":{"mlx5_0":{"port_xmit_data":380085435825712,"port_rcv_data":380488865202960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872870299600,"port_rcv_data":1408212417039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428739444838,"port_rcv_data":380150942987836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448560566144,"port_rcv_data":1038740093894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.640","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.651","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.661","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.671","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.680","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":105.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:11.688","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":110.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.697","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":107.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.706","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":84.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.5214837,"monotonic":2350060.968497756,"hcas":{"mlx5_0":{"port_xmit_data":380085435825912,"port_rcv_data":380488865203108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872870299600,"port_rcv_data":1408212417039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428739444838,"port_rcv_data":380150942987836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448560566144,"port_rcv_data":1038740093894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.600","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.611","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.621","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.21,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.640","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.650","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.660","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.669","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.522327,"monotonic":2350061.969340905,"hcas":{"mlx5_0":{"port_xmit_data":380085556366261,"port_rcv_data":380488865331485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872990839456,"port_rcv_data":1408212545921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428739449518,"port_rcv_data":380150942989076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448560570824,"port_rcv_data":1038740095134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.653","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.664","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.675","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.16,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.685","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:13.695","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.704","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.713","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.5231388,"monotonic":2350062.970152575,"hcas":{"mlx5_0":{"port_xmit_data":380085556688749,"port_rcv_data":380488984648625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1872991175010,"port_rcv_data":1408331875278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379428874694245,"port_rcv_data":380150945216410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448685318874,"port_rcv_data":1038742270450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.639","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":141.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.651","gpu_util_pct":98.0,"memory_used_mib":72407.0,"power_w":119.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.660","gpu_util_pct":83.0,"memory_used_mib":72407.0,"power_w":139.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.669","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":142.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.678","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.685","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.692","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.699","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.523971,"monotonic":2350063.970984824,"hcas":{"mlx5_0":{"port_xmit_data":380085671836436,"port_rcv_data":380489098198519,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873106319724,"port_rcv_data":1408445381060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429006768125,"port_rcv_data":380150947718172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448827888148,"port_rcv_data":1038744831500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.628","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":286.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.640","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":277.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:15.647","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":284.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:15.653","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":286.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:15.658","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":164.98,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.664","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":164.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.671","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":161.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.678","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":166.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.5247386,"monotonic":2350064.971751906,"hcas":{"mlx5_0":{"port_xmit_data":380085671836736,"port_rcv_data":380489098198597,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873106320008,"port_rcv_data":1408445381137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429072691819,"port_rcv_data":380150948936272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1448886810092,"port_rcv_data":1038745943020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.660","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":226.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.670","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":206.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.679","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":224.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.687","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":198.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.694","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":263.54,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.703","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":256.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.713","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":270.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.720","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":273.19,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.5255423,"monotonic":2350065.972555953,"hcas":{"mlx5_0":{"port_xmit_data":380085671836736,"port_rcv_data":380489098198597,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873106320008,"port_rcv_data":1408445381137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429219104992,"port_rcv_data":380150951854474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449040224572,"port_rcv_data":1038748886021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.623","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.633","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.641","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.649","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.656","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":228.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":228.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.670","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":224.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":212.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.5263553,"monotonic":2350066.97336835,"hcas":{"mlx5_0":{"port_xmit_data":380085671836736,"port_rcv_data":380489098198597,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873106320008,"port_rcv_data":1408445381137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429307520992,"port_rcv_data":380150953451098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449128640572,"port_rcv_data":1038750472275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.609","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.621","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.632","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.643","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.653","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":107.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.663","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":107.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.672","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":108.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.681","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.5271919,"monotonic":2350067.974205254,"hcas":{"mlx5_0":{"port_xmit_data":380085671836736,"port_rcv_data":380489098198597,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873106320008,"port_rcv_data":1408445381137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429307520992,"port_rcv_data":380150953451098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449128640572,"port_rcv_data":1038750472275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.644","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.656","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.667","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.678","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.688","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.698","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.707","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.716","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.5279992,"monotonic":2350068.975013081,"hcas":{"mlx5_0":{"port_xmit_data":380085672181380,"port_rcv_data":380489217538707,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873106681795,"port_rcv_data":1408564738375,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429307520992,"port_rcv_data":380150953451098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449128640572,"port_rcv_data":1038750472275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.630","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":99.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.643","gpu_util_pct":93.0,"memory_used_mib":72407.0,"power_w":102.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.656","gpu_util_pct":94.0,"memory_used_mib":72407.0,"power_w":99.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.666","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":108.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:20.676","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.683","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.689","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.696","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.5288308,"monotonic":2350069.975844193,"hcas":{"mlx5_0":{"port_xmit_data":380085787376969,"port_rcv_data":380489335062099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873221871447,"port_rcv_data":1408682223148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429307520992,"port_rcv_data":380150953451098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449128640572,"port_rcv_data":1038750472275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.647","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":300.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.657","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":274.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.666","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":295.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.674","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":287.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.683","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":127.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.691","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":115.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.698","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":122.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.706","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":126.19,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.5295208,"monotonic":2350070.976533735,"hcas":{"mlx5_0":{"port_xmit_data":380085922488888,"port_rcv_data":380489454511092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873321285557,"port_rcv_data":1408801640980,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429342893329,"port_rcv_data":380150954176779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449164011780,"port_rcv_data":1038751209998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.631","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":329.86,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.642","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":307.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.652","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":322.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.662","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":325.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:22.671","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":234.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.680","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":249.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.687","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":249.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.694","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":254.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.5303588,"monotonic":2350071.977372134,"hcas":{"mlx5_0":{"port_xmit_data":380086037666377,"port_rcv_data":380489572451890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873436457300,"port_rcv_data":1408919563535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429502042439,"port_rcv_data":380150957303011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449320602420,"port_rcv_data":1038754242883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.630","gpu_util_pct":98.0,"memory_used_mib":72407.0,"power_w":209.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.641","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":220.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.649","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":229.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.658","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":226.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.666","gpu_util_pct":97.0,"memory_used_mib":83581.0,"power_w":323.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.674","gpu_util_pct":97.0,"memory_used_mib":83581.0,"power_w":309.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.682","gpu_util_pct":56.0,"memory_used_mib":83581.0,"power_w":293.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.690","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":305.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.5311975,"monotonic":2350072.978210985,"hcas":{"mlx5_0":{"port_xmit_data":380086037667247,"port_rcv_data":380489572452122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873436458446,"port_rcv_data":1408919563799,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429649507818,"port_rcv_data":380150960097460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449471809317,"port_rcv_data":1038757206948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.628","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":254.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.639","gpu_util_pct":24.0,"memory_used_mib":72407.0,"power_w":279.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.648","gpu_util_pct":38.0,"memory_used_mib":72407.0,"power_w":254.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.657","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":231.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.665","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":270.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.673","gpu_util_pct":44.0,"memory_used_mib":83581.0,"power_w":258.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.681","gpu_util_pct":86.0,"memory_used_mib":83581.0,"power_w":248.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.689","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":232.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.5320437,"monotonic":2350073.979057112,"hcas":{"mlx5_0":{"port_xmit_data":380086037669871,"port_rcv_data":380489572452820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873436461739,"port_rcv_data":1408919564574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429802940541,"port_rcv_data":380150962915428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449627226213,"port_rcv_data":1038760129193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.642","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.650","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:25.658","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:25.666","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":320.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:25.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":293.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.681","gpu_util_pct":2.0,"memory_used_mib":83581.0,"power_w":266.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.687","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":305.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.5328934,"monotonic":2350074.979906794,"hcas":{"mlx5_0":{"port_xmit_data":380086298249469,"port_rcv_data":380489572731142,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873659840788,"port_rcv_data":1408919803444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379429946341104,"port_rcv_data":380150965479066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449765553784,"port_rcv_data":1038762504661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.631","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.642","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.650","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.658","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.665","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.672","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.679","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.686","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.5334518,"monotonic":2350075.980465105,"hcas":{"mlx5_0":{"port_xmit_data":380086298249469,"port_rcv_data":380489572731142,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873659840788,"port_rcv_data":1408919803444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379430087806704,"port_rcv_data":380150967984843,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1449908922824,"port_rcv_data":1038765083132,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.668","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:27.679","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:27.687","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:27.695","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.702","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.709","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.716","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.722","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.5342689,"monotonic":2350076.981282512,"hcas":{"mlx5_0":{"port_xmit_data":380086298250754,"port_rcv_data":380489572731468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873659842512,"port_rcv_data":1408919803816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379430238389639,"port_rcv_data":380150970622075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450059505288,"port_rcv_data":1038767718349,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.648","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.658","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.666","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.674","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.682","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.689","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.696","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.702","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.5351017,"monotonic":2350077.982115396,"hcas":{"mlx5_0":{"port_xmit_data":380086298253309,"port_rcv_data":380489572732134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873659845589,"port_rcv_data":1408919804545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379430381235740,"port_rcv_data":380150973119652,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450202350723,"port_rcv_data":1038770227334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.632","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.642","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.650","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.658","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.666","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.680","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.687","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.5359507,"monotonic":2350078.98296419,"hcas":{"mlx5_0":{"port_xmit_data":380086298600425,"port_rcv_data":380489692074034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873660182080,"port_rcv_data":1409039135034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379430533199856,"port_rcv_data":380150975765099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450354314189,"port_rcv_data":1038772874161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.675","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":171.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.687","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":181.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.696","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":167.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.705","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":160.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.714","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.721","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.727","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.734","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.5367522,"monotonic":2350079.983765885,"hcas":{"mlx5_0":{"port_xmit_data":380086413765258,"port_rcv_data":380489806881647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873775342969,"port_rcv_data":1409153922982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379430650143180,"port_rcv_data":380150977874742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450471256895,"port_rcv_data":1038774997459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.643","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":281.38,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.656","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":279.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.665","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":274.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.673","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":287.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.681","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":156.24,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.690","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":140.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.698","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":157.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.705","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":156.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.537584,"monotonic":2350080.98459754,"hcas":{"mlx5_0":{"port_xmit_data":380086413766529,"port_rcv_data":380489806881988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873775344323,"port_rcv_data":1409153923338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379430766396181,"port_rcv_data":380150980041318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450587509488,"port_rcv_data":1038777142706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.675","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":159.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.685","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":150.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.693","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":158.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.701","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":150.11,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.709","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":272.93,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.717","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":224.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.725","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":251.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.733","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":255.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.5383887,"monotonic":2350081.985402139,"hcas":{"mlx5_0":{"port_xmit_data":380086413766529,"port_rcv_data":380489806881988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873775344323,"port_rcv_data":1409153923338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379430916703691,"port_rcv_data":380150982824039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450735216288,"port_rcv_data":1038779781581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.675","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.683","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.689","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:33.694","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.700","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":190.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.705","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":211.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.710","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":203.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.716","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":214.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.5392182,"monotonic":2350082.986231754,"hcas":{"mlx5_0":{"port_xmit_data":380086532512295,"port_rcv_data":380489807008815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873894089267,"port_rcv_data":1409154049322,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431005124371,"port_rcv_data":380150984376844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450826237368,"port_rcv_data":1038781435167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.646","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.655","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.665","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.78,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.674","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.683","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.692","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.2,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.701","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.5400624,"monotonic":2350083.987075633,"hcas":{"mlx5_0":{"port_xmit_data":380086532513681,"port_rcv_data":380489807009171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873894090913,"port_rcv_data":1409154049710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431109772468,"port_rcv_data":380150986187244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1450930885124,"port_rcv_data":1038783279667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.676","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.687","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:35.695","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:35.702","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.710","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.717","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.723","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.730","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.5408797,"monotonic":2350084.987893193,"hcas":{"mlx5_0":{"port_xmit_data":380086532516236,"port_rcv_data":380489807009838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873894093991,"port_rcv_data":1409154050438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431258904431,"port_rcv_data":380150988743892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451080016421,"port_rcv_data":1038785879157,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.656","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.666","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.674","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.682","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.690","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.697","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.704","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.710","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.5417218,"monotonic":2350085.988735571,"hcas":{"mlx5_0":{"port_xmit_data":380086532516975,"port_rcv_data":380489807010055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873894094991,"port_rcv_data":1409154050686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431285843483,"port_rcv_data":380150989193283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451106955180,"port_rcv_data":1038786338081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.639","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.650","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:37.659","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:37.668","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.677","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.686","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.694","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.703","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.542555,"monotonic":2350086.989568761,"hcas":{"mlx5_0":{"port_xmit_data":380086532855987,"port_rcv_data":380489926344672,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1873894425273,"port_rcv_data":1409273377488,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431285843483,"port_rcv_data":380150989193283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451106955180,"port_rcv_data":1038786338081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.656","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":133.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.666","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":153.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.677","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":136.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.686","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":154.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.694","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.701","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.707","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.714","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.5433843,"monotonic":2350087.990398353,"hcas":{"mlx5_0":{"port_xmit_data":380086648035939,"port_rcv_data":380490044613507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874009566409,"port_rcv_data":1409391587167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431285843483,"port_rcv_data":380150989193283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451106955180,"port_rcv_data":1038786338081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.642","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":252.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.651","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":293.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.659","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":256.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.668","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":258.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.676","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":231.72,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.683","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":268.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.691","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":248.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.698","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":263.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.544237,"monotonic":2350088.991250105,"hcas":{"mlx5_0":{"port_xmit_data":380086648035939,"port_rcv_data":380490044613507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874009566409,"port_rcv_data":1409391587167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431375281616,"port_rcv_data":380150990822893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451186529580,"port_rcv_data":1038787980740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.662","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":127.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.673","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":136.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.681","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":147.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:40.689","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":127.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:40.697","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":231.66,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.704","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":235.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.712","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":243.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.719","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":229.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.5450137,"monotonic":2350089.992027293,"hcas":{"mlx5_0":{"port_xmit_data":380086648035939,"port_rcv_data":380490044613507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874009566409,"port_rcv_data":1409391587167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431506884007,"port_rcv_data":380150993457759,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451327995180,"port_rcv_data":1038790675771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.624","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.635","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.643","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.651","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.659","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":169.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.666","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":167.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.673","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":164.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.680","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":173.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.5458827,"monotonic":2350090.992895748,"hcas":{"mlx5_0":{"port_xmit_data":380086648035939,"port_rcv_data":380490044613507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874009566409,"port_rcv_data":1409391587167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431568775207,"port_rcv_data":380150994583261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451389886380,"port_rcv_data":1038791808542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.660","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.673","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.682","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.691","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.700","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":84.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.708","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":107.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.716","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":99.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.725","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":103.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.5467138,"monotonic":2350091.993727245,"hcas":{"mlx5_0":{"port_xmit_data":380086648035939,"port_rcv_data":380490044613507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874009566409,"port_rcv_data":1409391587167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431568775207,"port_rcv_data":380150994583261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451389886380,"port_rcv_data":1038791808542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.647","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.657","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.667","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.676","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.685","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.694","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.702","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.711","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.547561,"monotonic":2350092.994574564,"hcas":{"mlx5_0":{"port_xmit_data":380086648361025,"port_rcv_data":380490163934540,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874009898768,"port_rcv_data":1409510916000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431568775207,"port_rcv_data":380150994583261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451389886380,"port_rcv_data":1038791808542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.650","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":123.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.662","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":138.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.674","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":121.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.686","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":138.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.699","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.708","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.717","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.726","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.548377,"monotonic":2350093.995390902,"hcas":{"mlx5_0":{"port_xmit_data":380086763513973,"port_rcv_data":380490279502337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874125056834,"port_rcv_data":1409626467201,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431568775207,"port_rcv_data":380150994583261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451389886380,"port_rcv_data":1038791808542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.656","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":300.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.669","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":275.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.682","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":305.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.693","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":297.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.703","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":185.52,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.713","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":150.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.724","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":162.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.733","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":164.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.5490608,"monotonic":2350094.996073678,"hcas":{"mlx5_0":{"port_xmit_data":380086903551279,"port_rcv_data":380490279651910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874227894742,"port_rcv_data":1409626581052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431639512836,"port_rcv_data":380150996039580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451460623924,"port_rcv_data":1038793219120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.674","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":155.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.689","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":169.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:46.701","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":175.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.712","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":132.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.722","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":256.68,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.733","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":251.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.745","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":244.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.755","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":254.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.5498912,"monotonic":2350095.996903481,"hcas":{"mlx5_0":{"port_xmit_data":380086903551279,"port_rcv_data":380490279651910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874227894742,"port_rcv_data":1409626581052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431768142859,"port_rcv_data":380150998340330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451590985444,"port_rcv_data":1038795642188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.671","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.686","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":82.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.698","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.708","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.718","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":171.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.728","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":181.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.738","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":203.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.747","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":175.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.5507188,"monotonic":2350096.997731214,"hcas":{"mlx5_0":{"port_xmit_data":380086903551949,"port_rcv_data":380490279652096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874227895527,"port_rcv_data":1409626581253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379431905010566,"port_rcv_data":380151000872875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451723333309,"port_rcv_data":1038797958198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.655","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.671","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.682","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.693","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.89,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.703","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":84.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.712","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.721","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.731","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.5515513,"monotonic":2350097.998564265,"hcas":{"mlx5_0":{"port_xmit_data":380086903554135,"port_rcv_data":380490279652654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874227898104,"port_rcv_data":1409626581858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432031098230,"port_rcv_data":380151003064901,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451849681979,"port_rcv_data":1038800189206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.655","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.670","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.682","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.693","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.703","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.713","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.722","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.732","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.5523932,"monotonic":2350098.999405995,"hcas":{"mlx5_0":{"port_xmit_data":380086903887911,"port_rcv_data":380490398980248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874228233695,"port_rcv_data":1409745911805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432129874932,"port_rcv_data":380151004792608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451950984524,"port_rcv_data":1038802010689,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.674","gpu_util_pct":99.0,"memory_used_mib":72407.0,"power_w":96.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.692","gpu_util_pct":90.0,"memory_used_mib":72407.0,"power_w":109.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.703","gpu_util_pct":93.0,"memory_used_mib":72407.0,"power_w":92.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.714","gpu_util_pct":97.0,"memory_used_mib":72407.0,"power_w":111.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.725","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.734","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.744","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.753","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.553015,"monotonic":2350100.000028469,"hcas":{"mlx5_0":{"port_xmit_data":380087019084492,"port_rcv_data":380490517921297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874343398200,"port_rcv_data":1409864795426,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432139061795,"port_rcv_data":380151004952351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451960171240,"port_rcv_data":1038802168277,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.658","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":285.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.670","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":277.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.682","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":283.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:51.694","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":306.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.705","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":141.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.716","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":139.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.726","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":130.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.736","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":152.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.5538568,"monotonic":2350101.000869289,"hcas":{"mlx5_0":{"port_xmit_data":380087137829961,"port_rcv_data":380490518049723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874462143144,"port_rcv_data":1409864921860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432174433088,"port_rcv_data":380151005727012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1451995542320,"port_rcv_data":1038802886160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:52.679","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":250.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:52.694","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":252.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:52.705","gpu_util_pct":2.0,"memory_used_mib":72407.0,"power_w":239.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.716","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":218.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.726","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":262.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.736","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":265.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.747","gpu_util_pct":99.0,"memory_used_mib":83581.0,"power_w":256.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.756","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":267.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.554692,"monotonic":2350102.001704495,"hcas":{"mlx5_0":{"port_xmit_data":380087171607248,"port_rcv_data":380490518089349,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874502557907,"port_rcv_data":1409864965244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432295320729,"port_rcv_data":380151007990060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452110484644,"port_rcv_data":1038805133520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.709","gpu_util_pct":3.0,"memory_used_mib":72407.0,"power_w":81.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.723","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.734","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.744","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.753","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":265.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.763","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":237.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.771","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":243.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":233.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.5554912,"monotonic":2350103.002504221,"hcas":{"mlx5_0":{"port_xmit_data":380087260165832,"port_rcv_data":380490518180644,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874584478127,"port_rcv_data":1409865053610,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432430775188,"port_rcv_data":380151010483173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452250424271,"port_rcv_data":1038807619581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.699","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.713","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.723","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.733","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.55,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.743","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.753","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.762","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.771","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.5562985,"monotonic":2350104.00331057,"hcas":{"mlx5_0":{"port_xmit_data":380087260167802,"port_rcv_data":380490518181156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874584480489,"port_rcv_data":1409865054168,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432553382566,"port_rcv_data":380151012673276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452374491066,"port_rcv_data":1038809825773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.675","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.690","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.701","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.711","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.721","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.730","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.739","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.748","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.5571074,"monotonic":2350105.004120633,"hcas":{"mlx5_0":{"port_xmit_data":380087260169988,"port_rcv_data":380490518181714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874584483067,"port_rcv_data":1409865054772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432668184335,"port_rcv_data":380151014708039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452489292315,"port_rcv_data":1038811825924,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.695","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.711","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:56.722","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.733","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.743","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.754","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.763","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.773","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.5578878,"monotonic":2350106.004901134,"hcas":{"mlx5_0":{"port_xmit_data":380087260172243,"port_rcv_data":380490518182303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874584485860,"port_rcv_data":1409865055423,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432811099617,"port_rcv_data":380151017217706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452632206948,"port_rcv_data":1038814320944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.692","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.703","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.711","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.719","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.727","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.734","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.741","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.748","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.5586963,"monotonic":2350107.005709687,"hcas":{"mlx5_0":{"port_xmit_data":380087260174299,"port_rcv_data":380490518182845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874584488291,"port_rcv_data":1409865056012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432946623973,"port_rcv_data":380151019578790,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452767730800,"port_rcv_data":1038816644735,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.692","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.703","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.711","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.719","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.726","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.734","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.741","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.748","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.5595036,"monotonic":2350108.006517009,"hcas":{"mlx5_0":{"port_xmit_data":380087260489759,"port_rcv_data":380490637494133,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874584815814,"port_rcv_data":1409984378429,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432992282477,"port_rcv_data":380151020351449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452813388896,"port_rcv_data":1038817410280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.658","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":158.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.669","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":163.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.679","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":182.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.688","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":167.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.696","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.703","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.710","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.716","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.5603523,"monotonic":2350109.007366074,"hcas":{"mlx5_0":{"port_xmit_data":380087375624455,"port_rcv_data":380490754519920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874699953257,"port_rcv_data":1410101394307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379432992282477,"port_rcv_data":380151020351449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452813388896,"port_rcv_data":1038817410280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:00.644","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":293.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:00.654","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":296.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.663","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":301.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.672","gpu_util_pct":100.0,"memory_used_mib":72407.0,"power_w":274.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.682","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":204.95,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.690","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":229.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.698","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":212.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.706","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":236.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.5612123,"monotonic":2350110.008225703,"hcas":{"mlx5_0":{"port_xmit_data":380087375624455,"port_rcv_data":380490754519920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874699953257,"port_rcv_data":1410101394307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433063015425,"port_rcv_data":380151021762630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1452884121696,"port_rcv_data":1038818833211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.697","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":205.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.708","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":178.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.717","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":178.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.725","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":191.21,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.732","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":277.94,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.741","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":283.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.749","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":268.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.756","gpu_util_pct":100.0,"memory_used_mib":83581.0,"power_w":271.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.5620341,"monotonic":2350111.009047302,"hcas":{"mlx5_0":{"port_xmit_data":380087496165101,"port_rcv_data":380490754650756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820493113,"port_rcv_data":1410101525096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433222169443,"port_rcv_data":380151024733981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453043275176,"port_rcv_data":1038821830635,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.659","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.670","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.678","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.686","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.77,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.694","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":196.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.701","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":172.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.708","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":172.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.715","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":167.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.562888,"monotonic":2350112.009901488,"hcas":{"mlx5_0":{"port_xmit_data":380087496166617,"port_rcv_data":380490754651128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820494906,"port_rcv_data":1410101525499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433372821333,"port_rcv_data":380151027363835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453193926710,"port_rcv_data":1038824442447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.696","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":81.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.707","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.715","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.723","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.730","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.737","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.744","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.751","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.5637102,"monotonic":2350113.010723444,"hcas":{"mlx5_0":{"port_xmit_data":380087496169388,"port_rcv_data":380490754651841,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498199,"port_rcv_data":1410101526274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433524508915,"port_rcv_data":380151029975972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453345613610,"port_rcv_data":1038827057917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.663","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.673","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.681","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.689","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.696","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.703","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.710","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.716","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.5645652,"monotonic":2350114.011578383,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.715","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.729","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:05.740","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.751","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.05,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.761","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.771","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.789","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.5653348,"monotonic":2350115.01234816,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.713","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.725","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.737","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.745","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.16,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.753","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.760","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.768","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.775","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":80.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.5661235,"monotonic":2350116.01313739,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.713","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.725","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.736","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.746","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.756","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":83.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.766","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.776","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.785","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.566896,"monotonic":2350117.01390983,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:08.709","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:08.720","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.730","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.741","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.751","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.760","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.769","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.778","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.567678,"monotonic":2350118.014691833,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.713","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.725","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.736","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.747","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.757","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.766","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.775","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.784","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.5684516,"monotonic":2350119.01546533,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.716","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.728","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.739","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.750","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.760","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.770","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.779","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.788","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.5690074,"monotonic":2350120.016021288,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.717","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:11.729","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.741","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.751","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.761","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.771","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.781","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.790","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.569777,"monotonic":2350121.016790795,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:12.716","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.728","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.739","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.750","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.760","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.770","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.789","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.5705483,"monotonic":2350122.017561807,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.711","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.722","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.733","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.743","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.753","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.762","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.771","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.571333,"monotonic":2350123.018346589,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.716","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.728","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.739","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.749","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.759","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.769","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.779","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.788","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.5721092,"monotonic":2350124.019122903,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.715","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.727","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.739","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.749","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.760","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.769","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.779","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.788","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.5725324,"monotonic":2350125.019546039,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:16.719","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:16.731","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:16.742","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.753","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.763","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.772","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.791","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.5733051,"monotonic":2350126.020318955,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.719","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.731","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.742","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.753","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.762","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.772","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.791","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.5740805,"monotonic":2350127.021094256,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:18.715","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:18.727","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.738","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.749","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.760","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.770","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.779","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.789","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.5748534,"monotonic":2350128.021867112,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.719","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.731","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.742","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.752","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.763","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.772","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.791","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.575629,"monotonic":2350129.022642925,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.719","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.731","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.742","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.753","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.763","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":82.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.772","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.791","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.5764048,"monotonic":2350130.023418678,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.700","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.712","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.723","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.734","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.744","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.755","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.764","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.773","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.5770218,"monotonic":2350131.024035495,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:22.725","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:22.737","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.748","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.760","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.770","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.790","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.799","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.5777974,"monotonic":2350132.024810947,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.717","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.729","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.741","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.752","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.762","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.773","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.792","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.5785787,"monotonic":2350133.025592523,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.721","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.733","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.743","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.754","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.763","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.773","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.791","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.5793614,"monotonic":2350134.026375355,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.729","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.740","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.752","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.762","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.772","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.791","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.800","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.5801413,"monotonic":2350135.027154873,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.724","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.735","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.747","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.757","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.767","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.777","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.786","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.796","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.580922,"monotonic":2350136.027935434,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:27.729","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.741","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.752","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.763","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.773","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.783","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.792","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.802","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.581021,"monotonic":2350137.028034825,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.704","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.716","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.727","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.738","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.748","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.758","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.767","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.776","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.5818248,"monotonic":2350138.028838408,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.705","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.717","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.727","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.737","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.747","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.756","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.765","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.774","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.5826292,"monotonic":2350139.029642848,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:30.728","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:30.739","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.750","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.761","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.771","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.789","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.798","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.5834067,"monotonic":2350140.030420407,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.729","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.741","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.752","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.763","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.774","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.784","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.793","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.803","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.584182,"monotonic":2350141.031195662,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.732","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.743","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.753","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.763","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.773","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.782","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.792","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.800","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.584959,"monotonic":2350142.031972835,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.729","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.740","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.751","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.761","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.771","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.790","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.799","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.5850115,"monotonic":2350143.032025146,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.721","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.733","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.744","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.754","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.764","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.774","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.783","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.792","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.585802,"monotonic":2350144.032815782,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.730","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.742","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.754","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.765","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.775","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.786","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.796","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.805","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.5865774,"monotonic":2350145.033591252,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.733","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.744","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.755","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.765","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.775","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.785","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.794","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.803","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.5873544,"monotonic":2350146.03436803,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.736","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.747","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.758","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.769","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.778","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.788","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.797","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.806","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.5881362,"monotonic":2350147.035149979,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.708","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.720","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.731","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.741","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.751","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.760","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.769","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.778","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.5889444,"monotonic":2350148.035958145,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.732","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.743","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.754","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.764","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.773","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.783","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.792","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:39.801","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.589746,"monotonic":2350149.036759548,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.737","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.748","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.759","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.770","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.780","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.790","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:40.799","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.809","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.5905259,"monotonic":2350150.037539768,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.736","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":78.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.747","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.758","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.768","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.778","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.787","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.796","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":77.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.805","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259302.5910566,"monotonic":2350151.038070287,"hcas":{"mlx5_0":{"port_xmit_data":380087496169781,"port_rcv_data":380490754651996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1874820498837,"port_rcv_data":1410101526460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":379433560358891,"port_rcv_data":380151030585448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":1453381463324,"port_rcv_data":1038827658898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:42.736","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:42.748","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":79.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:42.758","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:42.769","gpu_util_pct":0.0,"memory_used_mib":72407.0,"power_w":77.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:42.779","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:42.788","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:42.797","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":76.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:42.807","gpu_util_pct":0.0,"memory_used_mib":83581.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_5.jsonl

@@ -0,0 +1,459 @@
+{"wall_time":1788258844.5685775,"monotonic":2349729.49615428,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:04.650","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:04.672","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:04.688","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:04.703","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:04.717","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:04.731","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:04.745","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:04.758","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258845.5693808,"monotonic":2349730.496957457,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.711","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.728","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.34,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.744","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.760","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.775","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.790","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.804","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.818","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.5701153,"monotonic":2349731.497691733,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.690","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.708","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.12,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.724","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.739","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.753","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.766","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.779","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.792","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.5708818,"monotonic":2349732.498458407,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.711","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.728","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.744","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.759","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.773","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.787","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.800","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.813","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.571624,"monotonic":2349733.499200631,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.714","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.731","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.748","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.763","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.777","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.791","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.804","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.817","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.5723722,"monotonic":2349734.499948927,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.674","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.692","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.708","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.723","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.737","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.751","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.764","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.777","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.5731525,"monotonic":2349735.500729128,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.710","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.727","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.743","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.758","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.772","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.785","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.798","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.811","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.5739107,"monotonic":2349736.501487492,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.714","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.731","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.748","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.763","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.777","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.790","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.803","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.816","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.5746472,"monotonic":2349737.50222414,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.714","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.731","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.1,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.748","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.763","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.777","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.790","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.804","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.816","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.5753996,"monotonic":2349738.502976384,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.722","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":79.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.739","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.86,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.755","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.770","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.784","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.797","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.811","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.823","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.5761337,"monotonic":2349739.503710102,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.720","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.737","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.08,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.754","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.769","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.783","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.797","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.810","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.823","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.5764506,"monotonic":2349740.504027202,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.716","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.734","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.751","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.766","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.781","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.796","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.810","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.824","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.5771809,"monotonic":2349741.504757523,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.715","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.733","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.749","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.765","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.779","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.793","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.807","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.820","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.5779188,"monotonic":2349742.50549558,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.723","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.741","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.758","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.773","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.788","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.803","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.816","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.830","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.5786452,"monotonic":2349743.506222033,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.704","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.722","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.739","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.754","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.768","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.782","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.796","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.809","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.5794072,"monotonic":2349744.506983339,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.702","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.720","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.736","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.751","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.766","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.780","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.793","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.806","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.580169,"monotonic":2349745.507745488,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.722","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.740","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.756","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.771","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.786","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.799","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.813","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.826","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.5809155,"monotonic":2349746.508492062,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.768","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.786","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.802","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.818","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.832","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.846","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.860","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.873","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.581613,"monotonic":2349747.509189541,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.704","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.721","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.738","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.753","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.767","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.781","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.795","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.808","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.582377,"monotonic":2349748.509953492,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.723","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.758","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.791","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.823","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.841","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.857","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.872","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.887","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.7,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.5830603,"monotonic":2349749.510636871,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.702","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.720","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.736","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.751","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.765","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.779","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.792","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.805","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.583828,"monotonic":2349750.511404436,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.700","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.717","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.83,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.734","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.750","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.764","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.778","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.792","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.805","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.584572,"monotonic":2349751.512148685,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.686","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.703","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.720","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.735","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.750","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.764","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.777","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.790","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.585356,"monotonic":2349752.512932612,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.687","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.704","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.721","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.736","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.751","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.764","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.778","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.791","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.586142,"monotonic":2349753.513718503,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.703","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.720","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.737","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.751","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.766","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.780","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.793","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.806","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.5865002,"monotonic":2349754.514076678,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.723","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.740","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.756","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.770","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.784","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.798","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.811","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.823","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.587257,"monotonic":2349755.514833355,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.734","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.752","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.768","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.783","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.797","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.810","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.823","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.836","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.587992,"monotonic":2349756.515568749,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.730","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.747","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.83,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.763","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.778","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.793","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.806","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.819","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.832","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.588743,"monotonic":2349757.516319584,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.730","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":79.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.747","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.763","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.778","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.792","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.805","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.818","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.831","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.589484,"monotonic":2349758.517060602,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.711","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.729","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.745","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.760","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.775","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.790","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.803","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.816","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.5902414,"monotonic":2349759.517817937,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.730","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.748","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.764","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.779","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.794","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.808","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.821","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.834","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.590978,"monotonic":2349760.518554428,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.694","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.712","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.729","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.744","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.758","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.772","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.786","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.799","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.5917506,"monotonic":2349761.51932749,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.739","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.756","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.773","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.788","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.802","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.816","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.829","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.842","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.5924823,"monotonic":2349762.520059178,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.739","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.756","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.34,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.772","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.787","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.801","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.815","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.828","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:37.841","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.593215,"monotonic":2349763.520791632,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.711","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.729","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.34,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.746","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.761","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.775","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.789","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.803","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.816","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.5939765,"monotonic":2349764.521552993,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.735","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.752","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.768","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.782","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.796","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.810","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.823","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.836","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.594131,"monotonic":2349765.521707379,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.731","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.748","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.61,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.765","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.780","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.795","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.809","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.822","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.835","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.594872,"monotonic":2349766.522448437,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.714","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.732","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.749","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.765","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.780","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.794","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.808","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:41.821","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.5956278,"monotonic":2349767.523204003,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.743","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.761","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.778","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.793","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.808","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.822","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.836","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.850","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.5963488,"monotonic":2349768.523925359,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.742","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.761","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.83,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.778","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.794","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.809","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.824","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.838","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.851","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.597074,"monotonic":2349769.524650617,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.742","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.761","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.777","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.793","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.808","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.822","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.836","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.849","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.5978005,"monotonic":2349770.525377482,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.739","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.757","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.775","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.790","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.805","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.820","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.835","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.849","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.5985255,"monotonic":2349771.526102195,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.720","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.738","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.755","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.771","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.786","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.801","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.815","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.829","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.5992706,"monotonic":2349772.526847379,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.739","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.758","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.57,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.775","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.791","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.806","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.820","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.834","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:47.848","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.6000056,"monotonic":2349773.527582385,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.726","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.744","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.63,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.761","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.777","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.792","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.807","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.821","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.835","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.6007543,"monotonic":2349774.528331006,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.702","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.721","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.65,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.738","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.754","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.769","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.784","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.798","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.812","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.6015205,"monotonic":2349775.529097454,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.702","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.728","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.767","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.803","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.835","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.854","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.872","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.888","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.6022103,"monotonic":2349776.529786934,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.719","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.737","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.755","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.770","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.785","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.800","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.814","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:51.827","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.602963,"monotonic":2349777.530539588,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.723","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.741","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.759","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.775","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.790","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.805","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.819","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.832","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.6036754,"monotonic":2349778.531252115,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.723","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.741","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.757","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.773","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.788","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.802","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.816","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.830","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.6044476,"monotonic":2349779.532024158,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.722","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.741","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.61,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.758","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.774","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":79.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.789","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.803","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.817","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.831","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.6052005,"monotonic":2349780.532777388,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.742","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.761","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.07,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.778","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.794","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.809","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.824","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.838","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.851","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.605933,"monotonic":2349781.533509731,"hcas":{"mlx5_0":{"port_xmit_data":111389893958595,"port_rcv_data":112996318166531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563893833,"port_rcv_data":1491118357971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239402329,"port_rcv_data":112788092896511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267788262,"port_rcv_data":2267013944998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.723","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.741","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.758","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.774","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.789","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.803","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.817","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.831","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.6067038,"monotonic":2349782.534280009,"hcas":{"mlx5_0":{"port_xmit_data":111389893968019,"port_rcv_data":112996327739651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563903257,"port_rcv_data":1491127931091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239411737,"port_rcv_data":112788102469631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267797686,"port_rcv_data":2267023518118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.712","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.730","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.747","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.763","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.777","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.791","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.805","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.818","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.6074796,"monotonic":2349783.535056078,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.687","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.705","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.721","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.737","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.751","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.765","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.779","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.792","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.6082802,"monotonic":2349784.535856514,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.732","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.749","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.766","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.781","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.796","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.810","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.823","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.836","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.608467,"monotonic":2349785.536043624,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.702","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.720","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.737","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.752","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.767","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.781","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.794","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.807","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.609256,"monotonic":2349786.536832499,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.731","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.748","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.764","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.779","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.794","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.808","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.821","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.834","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.6100254,"monotonic":2349787.537601725,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.689","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.707","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.723","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.739","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.753","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.767","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.781","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.794","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.6108346,"monotonic":2349788.538411043,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.710","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.727","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.744","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.759","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.773","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.789","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.803","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.817","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.611619,"monotonic":2349789.539195365,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.690","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.708","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.725","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.740","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.755","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.769","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.782","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.795","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.612422,"monotonic":2349790.539998387,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.758","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.776","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.793","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.808","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.823","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.837","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.851","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.864","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.6131651,"monotonic":2349791.540741439,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.731","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.748","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.765","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.780","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.795","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.809","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.822","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.835","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.6139371,"monotonic":2349792.541513563,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.759","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.776","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.793","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.808","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.822","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.837","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.850","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.863","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.614235,"monotonic":2349793.541811501,"hcas":{"mlx5_0":{"port_xmit_data":111389893982000,"port_rcv_data":112996342099331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563917238,"port_rcv_data":1491142290771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239425687,"port_rcv_data":112788116829311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267811667,"port_rcv_data":2267037877798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.758","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.776","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.792","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.807","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.821","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.835","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.848","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.861","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.614981,"monotonic":2349794.542557418,"hcas":{"mlx5_0":{"port_xmit_data":111389893995981,"port_rcv_data":112996356459011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563931250,"port_rcv_data":1491156650451,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239439699,"port_rcv_data":112788131188991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267825679,"port_rcv_data":2267052237478,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.759","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.777","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.794","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.810","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.824","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.839","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.852","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.865","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.6157224,"monotonic":2349795.543298879,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.739","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.756","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.57,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.773","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.789","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.803","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.817","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.831","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.844","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.6164637,"monotonic":2349796.544040216,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.717","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.735","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.751","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.766","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.781","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.795","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:11.808","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:11.821","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.6172569,"monotonic":2349797.544833599,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.717","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.735","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.752","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.768","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.782","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.796","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.810","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.823","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.6180363,"monotonic":2349798.545612893,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.734","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.752","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.769","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.784","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.799","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.813","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.827","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.840","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.6187925,"monotonic":2349799.546369128,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.713","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.731","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.747","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.762","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.777","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.791","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.804","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.817","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.6195734,"monotonic":2349800.547150049,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.786","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.804","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.821","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.837","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.851","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.866","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.880","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:15.893","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.6202788,"monotonic":2349801.547855501,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.717","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.735","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.752","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.768","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.782","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.796","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.810","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.823","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.6210542,"monotonic":2349802.548630975,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.788","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.823","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.847","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.864","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.882","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.898","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.913","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.927","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.6217222,"monotonic":2349803.549298729,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.742","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.760","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.776","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.791","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.805","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.819","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.833","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.846","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.6224782,"monotonic":2349804.550054932,"hcas":{"mlx5_0":{"port_xmit_data":111389894009946,"port_rcv_data":112996370818691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563945246,"port_rcv_data":1491171010131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239453680,"port_rcv_data":112788145548671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267839676,"port_rcv_data":2267066597158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.743","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.761","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.54,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.778","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.793","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.808","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:19.822","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:19.834","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:19.850","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":81.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.6231804,"monotonic":2349805.550757258,"hcas":{"mlx5_0":{"port_xmit_data":111389894023927,"port_rcv_data":112996385178371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563959227,"port_rcv_data":1491185369811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239467692,"port_rcv_data":112788159908351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267853672,"port_rcv_data":2267080956838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.771","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.788","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.54,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.805","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.821","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.835","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.849","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.863","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.876","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.6239078,"monotonic":2349806.551484658,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.728","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.745","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.762","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.777","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.792","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.806","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:21.820","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:21.833","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.62468,"monotonic":2349807.552256729,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.743","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.761","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.777","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.792","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.807","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:22.821","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.834","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.847","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.625434,"monotonic":2349808.553010784,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.746","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.764","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.54,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.780","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.796","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.811","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.825","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.840","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.854","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.6261804,"monotonic":2349809.553756952,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.721","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.739","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.29,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.756","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.772","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.787","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.801","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:24.815","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:24.828","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.6269584,"monotonic":2349810.554535055,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.766","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.784","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.801","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.817","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.832","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.847","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.860","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.874","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.627695,"monotonic":2349811.555271742,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.750","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.768","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.785","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.800","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.815","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.829","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.843","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.858","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.7,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.628447,"monotonic":2349812.5560236,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.750","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.768","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":84.08,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.785","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.800","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.815","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.829","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.842","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:27.855","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.6292,"monotonic":2349813.556776439,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.729","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.747","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.764","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.778","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.790","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.801","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.813","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.825","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.6299865,"monotonic":2349814.557563289,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.750","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.768","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.784","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.799","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.814","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.827","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:29.841","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:29.854","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.6307433,"monotonic":2349815.558320082,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.766","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.784","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.54,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.800","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.815","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.829","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.843","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.857","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.870","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.6314864,"monotonic":2349816.559063144,"hcas":{"mlx5_0":{"port_xmit_data":111389894037924,"port_rcv_data":112996399538051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563973239,"port_rcv_data":1491199729491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239481658,"port_rcv_data":112788174268031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267867607,"port_rcv_data":2267095316518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.733","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.751","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.768","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.784","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.798","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.812","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.826","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.839","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.6322708,"monotonic":2349817.559847317,"hcas":{"mlx5_0":{"port_xmit_data":111389894051920,"port_rcv_data":112996413897731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802563987205,"port_rcv_data":1491214089171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239495654,"port_rcv_data":112788188627711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618267881588,"port_rcv_data":2267109676198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.751","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.769","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.785","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.801","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.815","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.829","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.842","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.855","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.6330314,"monotonic":2349818.560607941,"hcas":{"mlx5_0":{"port_xmit_data":111389894342793,"port_rcv_data":112996430745371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802564278698,"port_rcv_data":1491231000171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304239689497,"port_rcv_data":112788197469311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618268073075,"port_rcv_data":2267118517798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.755","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.773","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":83.78,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.791","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.808","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.823","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.838","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.852","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.866","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.6337848,"monotonic":2349819.56136118,"hcas":{"mlx5_0":{"port_xmit_data":111389894634168,"port_rcv_data":112996537206354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802564567773,"port_rcv_data":1491337467589,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304240111831,"port_rcv_data":112788311955903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618268492319,"port_rcv_data":2267233032640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.731","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.749","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.766","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.781","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.796","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.810","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:34.823","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:34.837","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.6345644,"monotonic":2349820.562141017,"hcas":{"mlx5_0":{"port_xmit_data":111389894795290,"port_rcv_data":112996546048023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802564725671,"port_rcv_data":1491346309258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304240425272,"port_rcv_data":112788329366981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618268803171,"port_rcv_data":2267250435393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.775","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.793","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.809","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.825","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.839","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.854","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.867","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.880","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.6353014,"monotonic":2349821.562878261,"hcas":{"mlx5_0":{"port_xmit_data":111389895254555,"port_rcv_data":112996571756037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565180984,"port_rcv_data":1491371992348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304240574413,"port_rcv_data":112788337937153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618268956435,"port_rcv_data":2267258997193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.757","gpu_util_pct":0.0,"memory_used_mib":79931.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.775","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":82.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.791","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.806","gpu_util_pct":0.0,"memory_used_mib":79897.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.820","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.834","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:36.847","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:36.860","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.636057,"monotonic":2349822.563633632,"hcas":{"mlx5_0":{"port_xmit_data":111389895710645,"port_rcv_data":112996592042205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565333426,"port_rcv_data":1491380553493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304252820947,"port_rcv_data":112788355393450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269257368,"port_rcv_data":2267276119716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.738","gpu_util_pct":70.0,"memory_used_mib":79931.0,"power_w":81.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.757","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":85.6,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.774","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.790","gpu_util_pct":85.0,"memory_used_mib":79897.0,"power_w":81.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.805","gpu_util_pct":100.0,"memory_used_mib":79889.0,"power_w":81.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.820","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":90.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.834","gpu_util_pct":100.0,"memory_used_mib":79917.0,"power_w":89.13,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.848","gpu_util_pct":99.0,"memory_used_mib":80019.0,"power_w":89.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.6368217,"monotonic":2349823.564398479,"hcas":{"mlx5_0":{"port_xmit_data":111389902731023,"port_rcv_data":112996899590801,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565333426,"port_rcv_data":1491380553493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304561038034,"port_rcv_data":112788362517045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269257368,"port_rcv_data":2267276119716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.741","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":126.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.759","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.776","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.792","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":126.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.806","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":118.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.821","gpu_util_pct":48.0,"memory_used_mib":79935.0,"power_w":118.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.834","gpu_util_pct":43.0,"memory_used_mib":79919.0,"power_w":118.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.848","gpu_util_pct":32.0,"memory_used_mib":80021.0,"power_w":121.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.637591,"monotonic":2349824.565167418,"hcas":{"mlx5_0":{"port_xmit_data":111389903501577,"port_rcv_data":112996933895354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565333426,"port_rcv_data":1491380553493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304594548659,"port_rcv_data":112788363273563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269257368,"port_rcv_data":2267276119716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.758","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":90.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.782","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":88.22,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.799","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":85.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.815","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":83.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.829","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.844","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.857","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.871","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":81.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.6381903,"monotonic":2349825.565765874,"hcas":{"mlx5_0":{"port_xmit_data":111389903511001,"port_rcv_data":112996943468474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565342819,"port_rcv_data":1491390126613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304594558176,"port_rcv_data":112788372846683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269266885,"port_rcv_data":2267285692836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.766","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.784","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.95,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.801","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.816","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.831","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.845","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:40.858","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.96,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.871","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":81.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.6389446,"monotonic":2349826.566520904,"hcas":{"mlx5_0":{"port_xmit_data":111389903524905,"port_rcv_data":112996957828154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565356738,"port_rcv_data":1491404486293,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304594572188,"port_rcv_data":112788387206363,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269280850,"port_rcv_data":2267300052516,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.740","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.758","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.96,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.774","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.790","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.805","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:41.819","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.833","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.846","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.6397204,"monotonic":2349827.5672972,"hcas":{"mlx5_0":{"port_xmit_data":111389903586269,"port_rcv_data":112997020053434,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565417963,"port_rcv_data":1491466711573,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304594632840,"port_rcv_data":112788449431643,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269341548,"port_rcv_data":2267362277796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.793","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.812","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.829","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.846","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:42.862","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.877","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.892","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.906","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.6404433,"monotonic":2349828.568019932,"hcas":{"mlx5_0":{"port_xmit_data":111389903628398,"port_rcv_data":112997063132474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565460185,"port_rcv_data":1491509790613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304594675155,"port_rcv_data":112788492510683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269383848,"port_rcv_data":2267405356836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.766","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.784","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.800","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.816","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.830","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.844","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.858","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.871","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.6411965,"monotonic":2349829.568773186,"hcas":{"mlx5_0":{"port_xmit_data":111389903680370,"port_rcv_data":112997115784634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565512141,"port_rcv_data":1491562442773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304594726785,"port_rcv_data":112788545162843,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269436017,"port_rcv_data":2267458008996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.787","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.822","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.54,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.847","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.865","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":81.01,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.881","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.896","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.910","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.923","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.6418974,"monotonic":2349830.569474167,"hcas":{"mlx5_0":{"port_xmit_data":111389903746498,"port_rcv_data":112997182796474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565577669,"port_rcv_data":1491629454613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304594792164,"port_rcv_data":112788612174683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269501350,"port_rcv_data":2267525020836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.788","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.806","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.54,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.823","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.839","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:45.854","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:45.869","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:45.883","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.896","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.6426263,"monotonic":2349831.57020309,"hcas":{"mlx5_0":{"port_xmit_data":111389903974814,"port_rcv_data":112997249076794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802565802710,"port_rcv_data":1491695734933,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304595016074,"port_rcv_data":112788678455003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618269723666,"port_rcv_data":2267591301156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.785","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.804","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.822","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.838","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.868","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.899","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.932","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.970","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.6432643,"monotonic":2349832.570841071,"hcas":{"mlx5_0":{"port_xmit_data":111389904380526,"port_rcv_data":112997343730344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802566208903,"port_rcv_data":1491790437377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304595435442,"port_rcv_data":112788773107253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618270148351,"port_rcv_data":2267686002428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.793","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.811","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.828","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.844","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.859","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.873","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.888","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.902","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.6439824,"monotonic":2349833.571558977,"hcas":{"mlx5_0":{"port_xmit_data":111389904786719,"port_rcv_data":112997433262119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802566613670,"port_rcv_data":1491879956914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304595684232,"port_rcv_data":112788853784998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618270394398,"port_rcv_data":2267766692347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.792","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.810","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.826","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.841","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:48.856","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.870","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.883","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.896","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.6447122,"monotonic":2349834.572288603,"hcas":{"mlx5_0":{"port_xmit_data":111389904853602,"port_rcv_data":112997500311896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802566680661,"port_rcv_data":1491947018961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304596249889,"port_rcv_data":112788947371557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618270958520,"port_rcv_data":2267860266700,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.792","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.810","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.827","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.843","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.858","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.872","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.886","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.899","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.6454384,"monotonic":2349835.573014976,"hcas":{"mlx5_0":{"port_xmit_data":111389905461233,"port_rcv_data":112997613956970,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802567294663,"port_rcv_data":1492060750223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304596499269,"port_rcv_data":112789043354268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618271210659,"port_rcv_data":2267956294436,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.764","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.782","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.800","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.818","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":81.01,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.834","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.849","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.864","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.878","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.6461878,"monotonic":2349836.573764593,"hcas":{"mlx5_0":{"port_xmit_data":111389905706407,"port_rcv_data":112997702602613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802567536683,"port_rcv_data":1492149474118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304596920546,"port_rcv_data":112789140860824,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618271626183,"port_rcv_data":2268053838955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.764","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.782","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.799","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:51.814","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:51.829","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.843","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.856","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.869","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.6469176,"monotonic":2349837.574494048,"hcas":{"mlx5_0":{"port_xmit_data":111389905950222,"port_rcv_data":112997786550275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802567775430,"port_rcv_data":1492233323022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304597313130,"port_rcv_data":112789233608552,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618272021185,"port_rcv_data":2268146569033,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.770","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.788","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.805","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.821","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":82.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.836","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.851","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.865","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.878","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.647676,"monotonic":2349838.575252335,"hcas":{"mlx5_0":{"port_xmit_data":111389906514887,"port_rcv_data":112997894810265,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802568339847,"port_rcv_data":1492341666870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304597572011,"port_rcv_data":112789324143328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618272287059,"port_rcv_data":2268237269223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.753","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.771","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.788","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.804","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":82.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.819","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.833","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.849","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.863","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.648455,"monotonic":2349839.576031429,"hcas":{"mlx5_0":{"port_xmit_data":111389906919732,"port_rcv_data":112997975325209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802568745838,"port_rcv_data":1492422174636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304598165397,"port_rcv_data":112789413499904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618272874509,"port_rcv_data":2268326618109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.774","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.792","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.81,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.809","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.825","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.839","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.853","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.867","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.880","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.6492054,"monotonic":2349840.576782107,"hcas":{"mlx5_0":{"port_xmit_data":111389907328467,"port_rcv_data":112998070718167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802569157565,"port_rcv_data":1492517640890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304598617687,"port_rcv_data":112789511599049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618273331837,"port_rcv_data":2268424845955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.775","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.793","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.810","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.826","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:55.840","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.854","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.868","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.881","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.6499803,"monotonic":2349841.577557027,"hcas":{"mlx5_0":{"port_xmit_data":111389907947335,"port_rcv_data":112998174786658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802569784896,"port_rcv_data":1492621741703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304599015121,"port_rcv_data":112789604119669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618273731752,"port_rcv_data":2268517343020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.787","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.805","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.822","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.838","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.852","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.866","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.880","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.893","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.6502714,"monotonic":2349842.57784788,"hcas":{"mlx5_0":{"port_xmit_data":111389908486007,"port_rcv_data":112998270067048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802570334821,"port_rcv_data":1492716972101,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304599399092,"port_rcv_data":112789690830325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618274122728,"port_rcv_data":2268604012040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.770","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.788","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":83.05,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.805","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.820","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.835","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.849","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.862","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.876","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.6510274,"monotonic":2349843.578604051,"hcas":{"mlx5_0":{"port_xmit_data":111389908898606,"port_rcv_data":112998367708698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802570749253,"port_rcv_data":1492814630639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304599974669,"port_rcv_data":112789797042189,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618274703079,"port_rcv_data":2268710231988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.752","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":81.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.770","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.787","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.802","gpu_util_pct":0.0,"memory_used_mib":79899.0,"power_w":80.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:58.817","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:58.830","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:58.844","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:58.857","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.6511397,"monotonic":2349844.578716174,"hcas":{"mlx5_0":{"port_xmit_data":111389911181786,"port_rcv_data":112998532022560,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802571006289,"port_rcv_data":1492894504404,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304686222176,"port_rcv_data":112789893857068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618275279586,"port_rcv_data":2268804984193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.757","gpu_util_pct":99.0,"memory_used_mib":79933.0,"power_w":87.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.775","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":90.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.792","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":87.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.808","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":87.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.824","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":86.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.838","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":99.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.853","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":89.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.867","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":98.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.651901,"monotonic":2349845.579477498,"hcas":{"mlx5_0":{"port_xmit_data":111389918727273,"port_rcv_data":112998938676421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802571565731,"port_rcv_data":1492994604054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113304992986047,"port_rcv_data":112789994955387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618275725196,"port_rcv_data":2268899046043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.754","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":126.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.773","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":126.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.790","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":122.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.806","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":123.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.821","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":121.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.836","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.850","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.864","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":124.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.6526701,"monotonic":2349846.580247073,"hcas":{"mlx5_0":{"port_xmit_data":111389926326287,"port_rcv_data":112999354770404,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802572145989,"port_rcv_data":1493102984394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113305300482579,"port_rcv_data":112790101526620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618276162977,"port_rcv_data":2268998581691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.797","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":124.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.815","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.13,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.832","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.848","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":125.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.864","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":119.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.878","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":121.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:01.892","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":120.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.906","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":120.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.6533954,"monotonic":2349847.580972034,"hcas":{"mlx5_0":{"port_xmit_data":111389933891756,"port_rcv_data":112999767346533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802572740925,"port_rcv_data":1493207375388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113305610104978,"port_rcv_data":112790204057950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618276592683,"port_rcv_data":2269094129997,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.755","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":127.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.773","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":126.16,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.790","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.805","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":123.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.820","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":123.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.834","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":125.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.848","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.861","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":126.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.6541712,"monotonic":2349848.581747736,"hcas":{"mlx5_0":{"port_xmit_data":111389941151882,"port_rcv_data":113000136608304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802572973379,"port_rcv_data":1493268805598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113305917859684,"port_rcv_data":112790299126218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618277332436,"port_rcv_data":2269182083899,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.755","gpu_util_pct":99.0,"memory_used_mib":79933.0,"power_w":126.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.773","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.789","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.808","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":124.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.824","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":122.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.838","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":126.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.853","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.867","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":124.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.6549373,"monotonic":2349849.582514032,"hcas":{"mlx5_0":{"port_xmit_data":111389948927036,"port_rcv_data":113000579412350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802573568145,"port_rcv_data":1493400426907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113306230784421,"port_rcv_data":112790426802414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618277675529,"port_rcv_data":2269302564568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.738","gpu_util_pct":87.0,"memory_used_mib":79933.0,"power_w":128.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.756","gpu_util_pct":87.0,"memory_used_mib":79935.0,"power_w":125.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.774","gpu_util_pct":89.0,"memory_used_mib":79919.0,"power_w":121.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.790","gpu_util_pct":87.0,"memory_used_mib":79899.0,"power_w":123.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.805","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":117.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.819","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.834","gpu_util_pct":87.0,"memory_used_mib":79919.0,"power_w":123.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.847","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":126.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.655728,"monotonic":2349850.583304664,"hcas":{"mlx5_0":{"port_xmit_data":111389960704465,"port_rcv_data":113001075407232,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802573883306,"port_rcv_data":1493419263274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113306708496266,"port_rcv_data":112790459536503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618278087503,"port_rcv_data":2269323692431,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.758","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":129.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.777","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":129.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.794","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":129.31,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.810","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":127.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.825","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":124.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:05.840","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":125.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.854","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":126.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.867","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":125.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.656504,"monotonic":2349851.584080405,"hcas":{"mlx5_0":{"port_xmit_data":111389972716941,"port_rcv_data":113001583099906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802574252175,"port_rcv_data":1493441157178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113307194996227,"port_rcv_data":112790493014604,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618278458062,"port_rcv_data":2269345587507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.739","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":133.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.758","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":128.03,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.774","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":128.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.791","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":127.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.805","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":123.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.820","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":125.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.834","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.848","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":124.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.6572976,"monotonic":2349852.584874316,"hcas":{"mlx5_0":{"port_xmit_data":111389984834420,"port_rcv_data":113002096763378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802574717253,"port_rcv_data":1493467681978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113307682388406,"port_rcv_data":112790522438552,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618278771999,"port_rcv_data":2269363270707,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.795","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":129.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.813","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":129.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.830","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":128.86,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.846","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":127.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.862","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":121.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.876","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.94,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.891","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.904","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":126.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.6578999,"monotonic":2349853.585476165,"hcas":{"mlx5_0":{"port_xmit_data":111389996722076,"port_rcv_data":113002596612443,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802574866642,"port_rcv_data":1493476385502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113308174343517,"port_rcv_data":112790568841950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618279377832,"port_rcv_data":2269397877543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.759","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":130.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.777","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":131.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.795","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":128.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.811","gpu_util_pct":100.0,"memory_used_mib":79899.0,"power_w":126.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:08.826","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":123.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.841","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.855","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":126.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.868","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":124.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.6579702,"monotonic":2349854.585546697,"hcas":{"mlx5_0":{"port_xmit_data":111390009367928,"port_rcv_data":113003236689592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802575472351,"port_rcv_data":1493511337122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113308779636784,"port_rcv_data":112790581182603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618279378452,"port_rcv_data":2269398085423,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.821","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":135.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.839","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":132.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.857","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":128.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.873","gpu_util_pct":100.0,"memory_used_mib":79901.0,"power_w":130.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.888","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":123.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.903","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":133.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.917","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":128.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.930","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":128.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.658681,"monotonic":2349855.586257544,"hcas":{"mlx5_0":{"port_xmit_data":111390021704944,"port_rcv_data":113003885639412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802575627971,"port_rcv_data":1493520178722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113309420548232,"port_rcv_data":112790628783483,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618280007783,"port_rcv_data":2269433451823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.759","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":135.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.777","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":133.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.795","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":131.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.810","gpu_util_pct":100.0,"memory_used_mib":79901.0,"power_w":131.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.826","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":122.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.840","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":132.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.854","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":128.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.868","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":126.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.6594532,"monotonic":2349856.587029899,"hcas":{"mlx5_0":{"port_xmit_data":111390034539734,"port_rcv_data":113004564131717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802576100302,"port_rcv_data":1493546703522,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113310072886829,"port_rcv_data":112790658900769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618280325471,"port_rcv_data":2269451135023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.762","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":135.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.780","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":133.43,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.797","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":130.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.813","gpu_util_pct":100.0,"memory_used_mib":79901.0,"power_w":130.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.847","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":127.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.886","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":132.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:11.919","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":126.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.953","gpu_util_pct":99.0,"memory_used_mib":80023.0,"power_w":128.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.6601293,"monotonic":2349857.587706053,"hcas":{"mlx5_0":{"port_xmit_data":111390047132489,"port_rcv_data":113005230199302,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802576425244,"port_rcv_data":1493566979322,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113310717372365,"port_rcv_data":112790688880085,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618280633549,"port_rcv_data":2269468818223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.779","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":133.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.798","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":135.11,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.815","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":131.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.831","gpu_util_pct":100.0,"memory_used_mib":79901.0,"power_w":131.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:12.846","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":125.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:12.863","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":131.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.879","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":128.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.892","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":129.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.6604514,"monotonic":2349858.588028032,"hcas":{"mlx5_0":{"port_xmit_data":111390059265524,"port_rcv_data":113005998411479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802576955794,"port_rcv_data":1493598621693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113311456363555,"port_rcv_data":112790719347920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618280971433,"port_rcv_data":2269487607515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.762","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":139.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.781","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":134.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.798","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.813","gpu_util_pct":100.0,"memory_used_mib":79903.0,"power_w":130.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.828","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":125.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.843","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.857","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.870","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":130.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.6612222,"monotonic":2349859.588798662,"hcas":{"mlx5_0":{"port_xmit_data":111390070831089,"port_rcv_data":113006768674642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802577039850,"port_rcv_data":1493600996331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113312224018971,"port_rcv_data":112790766773693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618281590395,"port_rcv_data":2269523526768,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.744","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":136.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.762","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":134.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.779","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.794","gpu_util_pct":100.0,"memory_used_mib":79903.0,"power_w":130.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.809","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":126.42,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.824","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.838","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.851","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":132.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.6620147,"monotonic":2349860.589591333,"hcas":{"mlx5_0":{"port_xmit_data":111390083230774,"port_rcv_data":113007595029471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802577654472,"port_rcv_data":1493636776704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113313015255191,"port_rcv_data":112790787473267,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618281741287,"port_rcv_data":2269532369242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.762","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":142.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.780","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":137.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.798","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.814","gpu_util_pct":100.0,"memory_used_mib":79903.0,"power_w":131.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.829","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":132.45,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:15.844","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":135.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:15.858","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.872","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":133.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.6627798,"monotonic":2349861.590356683,"hcas":{"mlx5_0":{"port_xmit_data":111390095094524,"port_rcv_data":113008385849069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802577955900,"port_rcv_data":1493654252754,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113313789248030,"port_rcv_data":112790825607347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282200707,"port_rcv_data":2269558894292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.746","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":137.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.764","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":139.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.781","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.796","gpu_util_pct":100.0,"memory_used_mib":79903.0,"power_w":132.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.811","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":130.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.826","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":132.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.841","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.854","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":131.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.6635695,"monotonic":2349862.591146182,"hcas":{"mlx5_0":{"port_xmit_data":111390106964597,"port_rcv_data":113009180496405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113314571672593,"port_rcv_data":112790838741203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.767","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":128.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.785","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":137.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.802","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.818","gpu_util_pct":100.0,"memory_used_mib":79903.0,"power_w":123.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.833","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":128.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.848","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":133.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.862","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":127.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.876","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":131.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.664337,"monotonic":2349863.59191343,"hcas":{"mlx5_0":{"port_xmit_data":111390117646966,"port_rcv_data":113009981427927,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113315373634966,"port_rcv_data":112790849480582,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.768","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.786","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.803","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.819","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.833","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.861","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.875","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.6651108,"monotonic":2349864.592687441,"hcas":{"mlx5_0":{"port_xmit_data":111390127927248,"port_rcv_data":113010783995304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113316176321489,"port_rcv_data":112790859799960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.767","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.786","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.802","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.818","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.833","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.862","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.876","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.6658785,"monotonic":2349865.593455243,"hcas":{"mlx5_0":{"port_xmit_data":111390138139202,"port_rcv_data":113011658777914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113317050539716,"port_rcv_data":112790870044864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.782","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.800","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.818","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.834","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.849","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:20.864","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.878","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.892","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.666634,"monotonic":2349866.59421078,"hcas":{"mlx5_0":{"port_xmit_data":111390148312722,"port_rcv_data":113012576730680,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113317968033960,"port_rcv_data":112790880255108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.766","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.784","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.801","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.817","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.832","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.846","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.860","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.874","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.6674032,"monotonic":2349867.594980045,"hcas":{"mlx5_0":{"port_xmit_data":111390157681581,"port_rcv_data":113013476324376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113318869875433,"port_rcv_data":112790889681700,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.769","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.786","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.802","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.815","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.827","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:22.839","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.860","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.6681988,"monotonic":2349868.595775575,"hcas":{"mlx5_0":{"port_xmit_data":111390166088222,"port_rcv_data":113014390330293,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113319783651209,"port_rcv_data":112790898014512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.767","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.781","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.794","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.806","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.817","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.828","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.838","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.851","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.6690025,"monotonic":2349869.596579351,"hcas":{"mlx5_0":{"port_xmit_data":111390174123590,"port_rcv_data":113015257718395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113320648865615,"port_rcv_data":112790905924053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.767","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":129.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.780","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.793","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.805","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:24.816","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.22,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.826","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.836","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.845","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.6698153,"monotonic":2349870.597392185,"hcas":{"mlx5_0":{"port_xmit_data":111390182553322,"port_rcv_data":113016173712782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113321565339077,"port_rcv_data":112790914277983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.768","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.781","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.794","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.805","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.816","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.836","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.845","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.670241,"monotonic":2349871.597817589,"hcas":{"mlx5_0":{"port_xmit_data":111390190350162,"port_rcv_data":113017012012031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113322403029126,"port_rcv_data":112790921963351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.785","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":134.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.798","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.811","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.823","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":129.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:26.834","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":124.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.845","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.855","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.865","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.671044,"monotonic":2349872.598620577,"hcas":{"mlx5_0":{"port_xmit_data":111390199031458,"port_rcv_data":113017961630038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578126555,"port_rcv_data":1493664649290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113323354356077,"port_rcv_data":112790930610353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282211526,"port_rcv_data":2269560243879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.772","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":131.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.785","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.798","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.810","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.821","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.832","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.842","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.852","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.671127,"monotonic":2349873.598703577,"hcas":{"mlx5_0":{"port_xmit_data":111390207946458,"port_rcv_data":113018875998454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578136041,"port_rcv_data":1493674222410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113324257332177,"port_rcv_data":112790949007221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282220904,"port_rcv_data":2269569816999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.768","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":134.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.781","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:28.794","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:28.806","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:28.818","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:28.828","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.839","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.848","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.671935,"monotonic":2349874.599511913,"hcas":{"mlx5_0":{"port_xmit_data":111390216883732,"port_rcv_data":113019784737512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578149991,"port_rcv_data":1493688582090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113325154287352,"port_rcv_data":112790972233139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282234931,"port_rcv_data":2269584176679,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.789","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.802","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.815","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.827","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.838","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.849","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.859","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.868","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.672729,"monotonic":2349875.600305807,"hcas":{"mlx5_0":{"port_xmit_data":111390226602673,"port_rcv_data":113020772600881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578164143,"port_rcv_data":1493702941770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113326125623815,"port_rcv_data":112790996259197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282248974,"port_rcv_data":2269598536359,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.753","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.766","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.779","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.790","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:30.800","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:30.810","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.820","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.830","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.89,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.6732614,"monotonic":2349876.600838016,"hcas":{"mlx5_0":{"port_xmit_data":111390235810038,"port_rcv_data":113021718974201,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578192151,"port_rcv_data":1493731661130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113327042719872,"port_rcv_data":112791034091767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282276905,"port_rcv_data":2269627255719,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.772","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.786","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.799","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.812","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.824","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.845","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.856","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.6740773,"monotonic":2349877.601653837,"hcas":{"mlx5_0":{"port_xmit_data":111390244959608,"port_rcv_data":113022652854129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578206117,"port_rcv_data":1493746020810,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113327964317169,"port_rcv_data":112791057566483,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282290855,"port_rcv_data":2269641615399,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.772","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.785","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.797","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:32.809","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:32.820","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.849","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.674901,"monotonic":2349878.602477531,"hcas":{"mlx5_0":{"port_xmit_data":111390254655717,"port_rcv_data":113023637381336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578220129,"port_rcv_data":1493760380490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113328932349267,"port_rcv_data":112791081539542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282304883,"port_rcv_data":2269655975079,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.772","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.785","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.800","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.813","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.824","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.845","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.855","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.6757123,"monotonic":2349879.603288786,"hcas":{"mlx5_0":{"port_xmit_data":111390264064843,"port_rcv_data":113024571069848,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578389327,"port_rcv_data":1493769222090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113329859375916,"port_rcv_data":112791090754193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282304883,"port_rcv_data":2269655975079,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.793","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.807","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.831","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:34.843","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.853","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.863","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.873","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.6765108,"monotonic":2349880.604087551,"hcas":{"mlx5_0":{"port_xmit_data":111390273444209,"port_rcv_data":113025560511642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802578810493,"port_rcv_data":1493857876568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113330761242855,"port_rcv_data":112791188273037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618282741502,"port_rcv_data":2269744627765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.775","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.789","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.802","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.814","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.825","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.836","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.846","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.856","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.6773307,"monotonic":2349881.604907434,"hcas":{"mlx5_0":{"port_xmit_data":111390283523899,"port_rcv_data":113026584403456,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579150842,"port_rcv_data":1493906065681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113331735886891,"port_rcv_data":112791245999522,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283092376,"port_rcv_data":2269792679144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.813","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.826","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.839","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.851","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.862","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.872","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.882","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.892","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.6781135,"monotonic":2349882.605690083,"hcas":{"mlx5_0":{"port_xmit_data":111390292883732,"port_rcv_data":113027505820777,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579459679,"port_rcv_data":1493924025045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113332640047474,"port_rcv_data":112791272954782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283399276,"port_rcv_data":2269810639094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.776","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":134.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.789","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.802","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.814","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.825","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.27,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.836","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.847","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.857","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.678936,"monotonic":2349883.606512867,"hcas":{"mlx5_0":{"port_xmit_data":111390302659295,"port_rcv_data":113028485149055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113333608756609,"port_rcv_data":112791300479818,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.806","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.819","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.831","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:38.843","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:38.854","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.865","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.874","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.884","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.6797328,"monotonic":2349884.607309319,"hcas":{"mlx5_0":{"port_xmit_data":111390310946380,"port_rcv_data":113029346843800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113334470917949,"port_rcv_data":112791308678593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.780","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":133.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.794","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.808","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.820","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.831","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.842","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.852","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.862","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.6805456,"monotonic":2349885.608122035,"hcas":{"mlx5_0":{"port_xmit_data":111390318715473,"port_rcv_data":113030180721429,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113335304686549,"port_rcv_data":112791316348286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.781","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":133.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.794","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.809","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.840","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.871","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.899","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.914","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.927","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.6812987,"monotonic":2349886.608875276,"hcas":{"mlx5_0":{"port_xmit_data":111390327498984,"port_rcv_data":113031145880893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113336271979129,"port_rcv_data":112791325097093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.762","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":134.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.775","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:41.788","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:41.799","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.811","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.821","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.831","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.841","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.6821306,"monotonic":2349887.609707497,"hcas":{"mlx5_0":{"port_xmit_data":111390335595276,"port_rcv_data":113032021243900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113337150933744,"port_rcv_data":112791333104616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.801","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.826","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.838","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.849","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.860","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.870","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.682903,"monotonic":2349888.610480022,"hcas":{"mlx5_0":{"port_xmit_data":111390343965543,"port_rcv_data":113032931515774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113338054713314,"port_rcv_data":112791341371080,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.797","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.810","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.823","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:43.835","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:43.846","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.857","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.867","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.877","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.6836867,"monotonic":2349889.611263562,"hcas":{"mlx5_0":{"port_xmit_data":111390352273946,"port_rcv_data":113033833403197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113338958977827,"port_rcv_data":112791349620452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.785","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.798","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.810","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.822","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.832","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.843","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.853","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.862","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.684496,"monotonic":2349890.612072756,"hcas":{"mlx5_0":{"port_xmit_data":111390360528581,"port_rcv_data":113034729577705,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113339855030283,"port_rcv_data":112791357786402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.765","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.778","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.791","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.803","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.814","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.825","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.836","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.845","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.6853209,"monotonic":2349891.61289784,"hcas":{"mlx5_0":{"port_xmit_data":111390368759209,"port_rcv_data":113035621737768,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113340746581616,"port_rcv_data":112791365919867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.784","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.797","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.809","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.821","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.832","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.842","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.851","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.860","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.6861327,"monotonic":2349892.613709634,"hcas":{"mlx5_0":{"port_xmit_data":111390377034760,"port_rcv_data":113036521611452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113341646494835,"port_rcv_data":112791374115230,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.821","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.848","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.860","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.871","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":123.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.882","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.898","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.910","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.6868849,"monotonic":2349893.614461669,"hcas":{"mlx5_0":{"port_xmit_data":111390385019668,"port_rcv_data":113037382980711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579611331,"port_rcv_data":1493933004796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113342507914600,"port_rcv_data":112791381993566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283705385,"port_rcv_data":2269828598829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.801","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":130.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.826","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:48.838","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:48.849","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.869","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.879","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.687677,"monotonic":2349894.615253656,"hcas":{"mlx5_0":{"port_xmit_data":111390394637501,"port_rcv_data":113038372684975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579621019,"port_rcv_data":1493942577916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113343486070409,"port_rcv_data":112791401143167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283714809,"port_rcv_data":2269838171949,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.787","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.800","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:49.813","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.826","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.837","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.857","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.867","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.6884844,"monotonic":2349895.616061273,"hcas":{"mlx5_0":{"port_xmit_data":111390403789708,"port_rcv_data":113039300537243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579635031,"port_rcv_data":1493956937596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113344399818097,"port_rcv_data":112791424587113,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283728759,"port_rcv_data":2269852531629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.788","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":134.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.801","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.814","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.826","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.837","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.847","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.857","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.867","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.68929,"monotonic":2349896.616866737,"hcas":{"mlx5_0":{"port_xmit_data":111390413270313,"port_rcv_data":113040279035026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579663101,"port_rcv_data":1493985656956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113345351222608,"port_rcv_data":112791462733192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283756768,"port_rcv_data":2269881250989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.787","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.801","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.815","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.827","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.838","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.858","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.868","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.6900992,"monotonic":2349897.617676146,"hcas":{"mlx5_0":{"port_xmit_data":111390422992317,"port_rcv_data":113041272817297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579677284,"port_rcv_data":1494000016636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113346330831195,"port_rcv_data":112791486766084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283770826,"port_rcv_data":2269895610669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.805","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.831","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.843","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.854","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.864","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.874","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.884","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.6908903,"monotonic":2349898.618466916,"hcas":{"mlx5_0":{"port_xmit_data":111390432358565,"port_rcv_data":113042225874386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579691280,"port_rcv_data":1494014376316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113347269406286,"port_rcv_data":112791510444194,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283784792,"port_rcv_data":2269909970349,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.787","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.801","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.813","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.825","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.837","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.862","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.873","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.691684,"monotonic":2349899.619260934,"hcas":{"mlx5_0":{"port_xmit_data":111390441907945,"port_rcv_data":113043198811829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579705587,"port_rcv_data":1494028735996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113348225703579,"port_rcv_data":112791534273226,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283798819,"port_rcv_data":2269924330029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.791","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.805","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:54.818","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.829","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.840","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.850","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.861","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.870","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.6924949,"monotonic":2349900.620071533,"hcas":{"mlx5_0":{"port_xmit_data":111390451527217,"port_rcv_data":113044156403005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802579888347,"port_rcv_data":1494037577596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113349176961763,"port_rcv_data":112791551173778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618283928477,"port_rcv_data":2269931808069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.772","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.786","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.798","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.810","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.822","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.832","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.842","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.852","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.6933224,"monotonic":2349901.620899407,"hcas":{"mlx5_0":{"port_xmit_data":111390461292187,"port_rcv_data":113045221241620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802580157985,"port_rcv_data":1494147758374,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113350132146673,"port_rcv_data":112791680953494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618284405985,"port_rcv_data":2270052192279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.791","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.805","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.818","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.830","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.840","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.861","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.870","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.6941364,"monotonic":2349902.621713091,"hcas":{"mlx5_0":{"port_xmit_data":111390471341936,"port_rcv_data":113046221337123,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802580473503,"port_rcv_data":1494165441574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113351113736781,"port_rcv_data":112791699490076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618284563326,"port_rcv_data":2270061033879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.792","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.805","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.818","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.830","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.841","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.852","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.862","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.872","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.694557,"monotonic":2349903.622133593,"hcas":{"mlx5_0":{"port_xmit_data":111390480726577,"port_rcv_data":113047144692264,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802580782821,"port_rcv_data":1494183400869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113352017765827,"port_rcv_data":112791717467561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618284716807,"port_rcv_data":2270070014069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.792","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.806","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.831","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.842","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.853","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.863","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.872","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.695375,"monotonic":2349904.622951565,"hcas":{"mlx5_0":{"port_xmit_data":111390490113885,"port_rcv_data":113048075910019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802580939526,"port_rcv_data":1494192173602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113352941133126,"port_rcv_data":112791753584173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285189169,"port_rcv_data":2270096953348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.796","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.810","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.823","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.834","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.845","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.855","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.865","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.875","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.6961937,"monotonic":2349905.623770423,"hcas":{"mlx5_0":{"port_xmit_data":111390499244925,"port_rcv_data":113048987360229,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581095069,"port_rcv_data":1494201360302,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113353844468634,"port_rcv_data":112791762505550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285189262,"port_rcv_data":2270096953779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.796","gpu_util_pct":98.0,"memory_used_mib":79957.0,"power_w":134.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.810","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.823","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.835","gpu_util_pct":99.0,"memory_used_mib":79923.0,"power_w":135.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.846","gpu_util_pct":99.0,"memory_used_mib":79915.0,"power_w":129.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.856","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.866","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.876","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.6970189,"monotonic":2349906.624595567,"hcas":{"mlx5_0":{"port_xmit_data":111390507826889,"port_rcv_data":113049914286163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581095069,"port_rcv_data":1494201360302,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113354771691790,"port_rcv_data":112791771021935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285189262,"port_rcv_data":2270096953779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.812","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.825","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.838","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:01.849","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.860","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":122.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.870","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.880","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.890","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.6978235,"monotonic":2349907.625400388,"hcas":{"mlx5_0":{"port_xmit_data":111390517096944,"port_rcv_data":113050847068946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581104493,"port_rcv_data":1494210933422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113355693260089,"port_rcv_data":112791789778890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285198686,"port_rcv_data":2270106526899,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.795","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.808","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.821","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.832","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.843","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.853","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.864","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.873","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.6986473,"monotonic":2349908.626223981,"hcas":{"mlx5_0":{"port_xmit_data":111390526502097,"port_rcv_data":113051804148706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581118489,"port_rcv_data":1494225293102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113356638027784,"port_rcv_data":112791813509181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285212714,"port_rcv_data":2270120886579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.800","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.813","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.825","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:03.837","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.847","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.858","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.868","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.878","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.699463,"monotonic":2349909.627039698,"hcas":{"mlx5_0":{"port_xmit_data":111390535916808,"port_rcv_data":113052776969455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581146668,"port_rcv_data":1494254012462,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113357582084798,"port_rcv_data":112791851578494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285240707,"port_rcv_data":2270149605939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.780","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.793","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.806","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.818","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.830","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.841","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.851","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.861","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.7002974,"monotonic":2349910.627874127,"hcas":{"mlx5_0":{"port_xmit_data":111390545055939,"port_rcv_data":113053702375839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581160727,"port_rcv_data":1494268372142,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113358492758097,"port_rcv_data":112791874985275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285254750,"port_rcv_data":2270163965619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.800","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.827","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.840","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.850","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.861","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.871","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.7011108,"monotonic":2349911.628687957,"hcas":{"mlx5_0":{"port_xmit_data":111390554042161,"port_rcv_data":113054610796660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581174708,"port_rcv_data":1494282731822,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113359386506057,"port_rcv_data":112791898249739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285268746,"port_rcv_data":2270178325299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.799","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.813","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.826","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.838","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.849","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.860","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.870","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.880","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.7019272,"monotonic":2349912.629503901,"hcas":{"mlx5_0":{"port_xmit_data":111390563837455,"port_rcv_data":113055616332049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581189464,"port_rcv_data":1494297091502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113360377761422,"port_rcv_data":112791922374265,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285282805,"port_rcv_data":2270192684979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.816","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.842","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.855","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.866","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.876","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.886","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.896","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.7027287,"monotonic":2349913.630305493,"hcas":{"mlx5_0":{"port_xmit_data":111390573376563,"port_rcv_data":113056561747463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581358972,"port_rcv_data":1494305933102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113361314711561,"port_rcv_data":112791940527852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285456017,"port_rcv_data":2270201526579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.800","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.813","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.826","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.838","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.850","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.861","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.7035463,"monotonic":2349914.631122802,"hcas":{"mlx5_0":{"port_xmit_data":111390583433635,"port_rcv_data":113057658428200,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802581624099,"port_rcv_data":1494414620272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113362303411377,"port_rcv_data":112792073924004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618285953319,"port_rcv_data":2270325375189,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.804","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.817","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:09.830","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:09.842","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.853","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.864","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.874","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.884","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.7043624,"monotonic":2349915.631939102,"hcas":{"mlx5_0":{"port_xmit_data":111390593555161,"port_rcv_data":113058657354852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582106583,"port_rcv_data":1494441144937,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113363273515157,"port_rcv_data":112792086175865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286050039,"port_rcv_data":2270327891324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.805","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.831","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.843","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.854","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.864","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.874","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.884","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.7044551,"monotonic":2349916.63203175,"hcas":{"mlx5_0":{"port_xmit_data":111390602928046,"port_rcv_data":113059588372798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582266698,"port_rcv_data":1494449986540,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113364198512189,"port_rcv_data":112792113061800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286357079,"port_rcv_data":2270345574763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.840","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.867","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.879","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.890","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.900","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.910","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.920","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.7052364,"monotonic":2349917.632813334,"hcas":{"mlx5_0":{"port_xmit_data":111390612552245,"port_rcv_data":113060540281854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582571878,"port_rcv_data":1494467462521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113365132956375,"port_rcv_data":112792140005927,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286673155,"port_rcv_data":2270363258201,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.804","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.831","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.843","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.855","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.866","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.877","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.889","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.706039,"monotonic":2349918.633615764,"hcas":{"mlx5_0":{"port_xmit_data":111390622004032,"port_rcv_data":113061499974081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582572265,"port_rcv_data":1494467669608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113366090643907,"port_rcv_data":112792149392856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286673201,"port_rcv_data":2270363258417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.827","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":142.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.857","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":141.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:13.887","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":139.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.907","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.920","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.932","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.943","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.953","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.7067788,"monotonic":2349919.634355535,"hcas":{"mlx5_0":{"port_xmit_data":111390630164061,"port_rcv_data":113062374316201,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582572265,"port_rcv_data":1494467669608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113366966511187,"port_rcv_data":112792157481785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286673201,"port_rcv_data":2270363258417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.821","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.834","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.847","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.859","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.870","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.881","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.891","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.900","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.707573,"monotonic":2349920.635149695,"hcas":{"mlx5_0":{"port_xmit_data":111390639794136,"port_rcv_data":113063353004599,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582581751,"port_rcv_data":1494477242728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113367935758217,"port_rcv_data":112792176651328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286682703,"port_rcv_data":2270372831537,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.825","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.838","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.851","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.863","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.875","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.897","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.909","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.7083113,"monotonic":2349921.635887961,"hcas":{"mlx5_0":{"port_xmit_data":111390649358657,"port_rcv_data":113064343819134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582609806,"port_rcv_data":1494505962088,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113368897450225,"port_rcv_data":112792214874686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286710882,"port_rcv_data":2270401550897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.789","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:16.802","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:16.815","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.827","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.838","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.858","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.868","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.7091393,"monotonic":2349922.636716317,"hcas":{"mlx5_0":{"port_xmit_data":111390658741603,"port_rcv_data":113065296413180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582623849,"port_rcv_data":1494520321768,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113369835795195,"port_rcv_data":112792238559877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286724878,"port_rcv_data":2270415910577,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.826","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.839","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.852","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.863","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.874","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.895","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.906","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.7099266,"monotonic":2349923.637503562,"hcas":{"mlx5_0":{"port_xmit_data":111390668009437,"port_rcv_data":113066237879415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582637861,"port_rcv_data":1494534681448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113370762918717,"port_rcv_data":112792262122310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286738921,"port_rcv_data":2270430270257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.808","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.821","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:18.834","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.845","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.856","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.877","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.886","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.7107391,"monotonic":2349924.638315904,"hcas":{"mlx5_0":{"port_xmit_data":111390677284043,"port_rcv_data":113067184411921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582651842,"port_rcv_data":1494549041128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113371695558566,"port_rcv_data":112792285703386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286752933,"port_rcv_data":2270444629937,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.825","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.838","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.851","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.863","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.873","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.904","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.7115357,"monotonic":2349925.63911238,"hcas":{"mlx5_0":{"port_xmit_data":111390686728730,"port_rcv_data":113068126686160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802582665963,"port_rcv_data":1494563400808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113372623036822,"port_rcv_data":112792309446467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286766914,"port_rcv_data":2270458989617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.792","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:20.805","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:20.818","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.830","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.841","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.861","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.870","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.7123656,"monotonic":2349926.639942288,"hcas":{"mlx5_0":{"port_xmit_data":111390697486657,"port_rcv_data":113069100323658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802583023749,"port_rcv_data":1494581084008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113373576992990,"port_rcv_data":112792328667648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618286939770,"port_rcv_data":2270467831217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.808","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.822","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.834","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.845","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.856","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.876","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.886","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.7131786,"monotonic":2349927.640755467,"hcas":{"mlx5_0":{"port_xmit_data":111390707642476,"port_rcv_data":113070116144876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802583129087,"port_rcv_data":1494679431666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113374496255295,"port_rcv_data":112792463514830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618287541093,"port_rcv_data":2270592701947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.811","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.825","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.837","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.849","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.860","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.871","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.881","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.890","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.7139897,"monotonic":2349928.64156648,"hcas":{"mlx5_0":{"port_xmit_data":111390718883944,"port_rcv_data":113071075027479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802583619476,"port_rcv_data":1494729750983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113375405983972,"port_rcv_data":112792506208301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618287693752,"port_rcv_data":2270624692045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.829","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.842","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.856","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.869","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.881","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.892","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.902","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.912","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.7147768,"monotonic":2349929.642353853,"hcas":{"mlx5_0":{"port_xmit_data":111390729978990,"port_rcv_data":113071970028833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802583787078,"port_rcv_data":1494752814187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113376276115275,"port_rcv_data":112792549591663,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618288028692,"port_rcv_data":2270657105248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.812","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.825","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.838","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.849","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.861","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.871","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.881","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.890","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.7155893,"monotonic":2349930.643166191,"hcas":{"mlx5_0":{"port_xmit_data":111390741308386,"port_rcv_data":113072882892130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584100395,"port_rcv_data":1494784580570,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113377159675094,"port_rcv_data":112792583718937,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618288193689,"port_rcv_data":2270680168823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.817","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.855","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.866","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.877","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.887","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.896","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.716397,"monotonic":2349931.643973943,"hcas":{"mlx5_0":{"port_xmit_data":111390751978676,"port_rcv_data":113073855106191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584128512,"port_rcv_data":1494813299930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113378103062611,"port_rcv_data":112792623092317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618288221775,"port_rcv_data":2270708888183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.816","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.829","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.842","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.854","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.864","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.875","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.885","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.895","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.717207,"monotonic":2349932.64478385,"hcas":{"mlx5_0":{"port_xmit_data":111390763251633,"port_rcv_data":113074966797594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584142524,"port_rcv_data":1494827659610,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113379198365996,"port_rcv_data":112792648750900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618288235942,"port_rcv_data":2270723247863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.833","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.846","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.859","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.871","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.882","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.893","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.903","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.912","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.7180011,"monotonic":2349933.645577907,"hcas":{"mlx5_0":{"port_xmit_data":111390774506542,"port_rcv_data":113076054969586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584313210,"port_rcv_data":1494836501210,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113380280046708,"port_rcv_data":112792659938301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618288235942,"port_rcv_data":2270723247863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.800","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.813","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.826","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:28.838","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.848","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.869","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.878","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.7188275,"monotonic":2349934.646404256,"hcas":{"mlx5_0":{"port_xmit_data":111390785830741,"port_rcv_data":113077166346238,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584512276,"port_rcv_data":1494884764384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113381341458778,"port_rcv_data":112792728134833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618288603664,"port_rcv_data":2270780350845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.816","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.843","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.854","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.865","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.876","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.886","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.895","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.719639,"monotonic":2349935.64721561,"hcas":{"mlx5_0":{"port_xmit_data":111390797612064,"port_rcv_data":113078196762540,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584856004,"port_rcv_data":1494955154342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113382301921283,"port_rcv_data":112792810752200,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618288974161,"port_rcv_data":2270851389253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.801","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.815","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.828","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.840","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.851","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.7204552,"monotonic":2349936.64803148,"hcas":{"mlx5_0":{"port_xmit_data":111390809151699,"port_rcv_data":113079188657146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584911820,"port_rcv_data":1494984914310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113383264139019,"port_rcv_data":112792851370457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618289004215,"port_rcv_data":2270880500835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.822","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.848","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.860","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.871","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.881","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.892","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.901","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.721271,"monotonic":2349937.648847638,"hcas":{"mlx5_0":{"port_xmit_data":111390820621430,"port_rcv_data":113080177800055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802584941394,"port_rcv_data":1495014025828,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113384224248548,"port_rcv_data":112792891936962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618289033774,"port_rcv_data":2270909612353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.821","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:32.848","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:32.860","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.871","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.882","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.892","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.901","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.722088,"monotonic":2349938.649664784,"hcas":{"mlx5_0":{"port_xmit_data":111390832222448,"port_rcv_data":113081160690073,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802585109677,"port_rcv_data":1495037088894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113385184564085,"port_rcv_data":112792926502331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618289201902,"port_rcv_data":2270932675712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.804","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.830","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.842","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.853","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.864","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.873","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.883","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.722917,"monotonic":2349939.650493874,"hcas":{"mlx5_0":{"port_xmit_data":111390843850304,"port_rcv_data":113082146018265,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802585275822,"port_rcv_data":1495060152029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113386146984226,"port_rcv_data":112792969781144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618289525806,"port_rcv_data":2270964442534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.838","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.864","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.875","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.887","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.897","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.907","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.917","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.723711,"monotonic":2349940.651287634,"hcas":{"mlx5_0":{"port_xmit_data":111390855628172,"port_rcv_data":113083136073692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802585636569,"port_rcv_data":1495091918412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113387107295049,"port_rcv_data":112793004345592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618289691950,"port_rcv_data":2270987506109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.824","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.837","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:35.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:35.863","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.874","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.904","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.724517,"monotonic":2349941.652094055,"hcas":{"mlx5_0":{"port_xmit_data":111390867066065,"port_rcv_data":113084208805949,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802585991426,"port_rcv_data":1495109601612,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113388163219788,"port_rcv_data":112793024341396,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618289864217,"port_rcv_data":2270996347709,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.824","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.838","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.862","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.873","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.905","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.7253265,"monotonic":2349942.652903486,"hcas":{"mlx5_0":{"port_xmit_data":111390878154796,"port_rcv_data":113085319669385,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802586028471,"port_rcv_data":1495144744706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113389239662539,"port_rcv_data":112793097003239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618290383576,"port_rcv_data":2271058014983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.824","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.838","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:37.851","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.863","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.874","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.885","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.895","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.905","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.7261393,"monotonic":2349943.653715658,"hcas":{"mlx5_0":{"port_xmit_data":111390889845114,"port_rcv_data":113086321284389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802586388939,"port_rcv_data":1495207012342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113390177583662,"port_rcv_data":112793153038493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618290429068,"port_rcv_data":2271102598247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.862","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.877","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.889","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.902","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.913","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.924","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.934","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.944","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.7269068,"monotonic":2349944.654483629,"hcas":{"mlx5_0":{"port_xmit_data":111390903382437,"port_rcv_data":113087327020679,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802586433129,"port_rcv_data":1495250908250,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113391138625753,"port_rcv_data":112793210472689,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618290473197,"port_rcv_data":2271146494155,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.842","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.855","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.868","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.880","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.891","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.902","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.912","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.923","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.7276976,"monotonic":2349945.655274478,"hcas":{"mlx5_0":{"port_xmit_data":111390916799177,"port_rcv_data":113088309309266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802586463292,"port_rcv_data":1495280451880,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113392091492544,"port_rcv_data":112793253461069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618290503685,"port_rcv_data":2271176037849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.845","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.871","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.884","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.896","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.907","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.917","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.927","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.7284822,"monotonic":2349946.656059302,"hcas":{"mlx5_0":{"port_xmit_data":111390930269914,"port_rcv_data":113089297036153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802586493626,"port_rcv_data":1495309995510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113393052068428,"port_rcv_data":112793296531814,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618290534189,"port_rcv_data":2271205581479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.810","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.824","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.837","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.850","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.861","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.871","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.881","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.891","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.7292986,"monotonic":2349947.656875681,"hcas":{"mlx5_0":{"port_xmit_data":111390944032861,"port_rcv_data":113090281199758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802586876181,"port_rcv_data":1495342314416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113394003745867,"port_rcv_data":112793332274699,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618290687174,"port_rcv_data":2271227915598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.848","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:42.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:42.876","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.890","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.903","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.914","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.925","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.935","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.7300744,"monotonic":2349948.657651168,"hcas":{"mlx5_0":{"port_xmit_data":111390957377090,"port_rcv_data":113091231667897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802586876305,"port_rcv_data":1495342314692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113394952944485,"port_rcv_data":112793373605831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618291221785,"port_rcv_data":2271255860628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.846","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.860","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.873","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.884","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.895","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.906","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.916","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.926","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.730862,"monotonic":2349949.658438684,"hcas":{"mlx5_0":{"port_xmit_data":111390971318671,"port_rcv_data":113092278353713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802587426493,"port_rcv_data":1495430511486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113395912133110,"port_rcv_data":112793450294387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618291289241,"port_rcv_data":2271319182619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.877","gpu_util_pct":98.0,"memory_used_mib":79957.0,"power_w":150.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.890","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.903","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.915","gpu_util_pct":99.0,"memory_used_mib":79923.0,"power_w":147.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.926","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.937","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.947","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.957","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.73162,"monotonic":2349950.659196871,"hcas":{"mlx5_0":{"port_xmit_data":111390983910278,"port_rcv_data":113093372772210,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802587628226,"port_rcv_data":1495484880548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113396953455754,"port_rcv_data":112793532535338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618291796261,"port_rcv_data":2271389010261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:45.832","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:45.845","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:45.859","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.871","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.883","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.895","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.905","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.915","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.732422,"monotonic":2349951.659999027,"hcas":{"mlx5_0":{"port_xmit_data":111390997067794,"port_rcv_data":113094361826986,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802587964250,"port_rcv_data":1495526496548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113397901970827,"port_rcv_data":112793578536274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618291989081,"port_rcv_data":2271422079021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.813","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.852","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.866","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.891","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.917","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.944","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.733171,"monotonic":2349952.660747778,"hcas":{"mlx5_0":{"port_xmit_data":111391009913875,"port_rcv_data":113095279880063,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802588007681,"port_rcv_data":1495569827048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113398775535495,"port_rcv_data":112793634695523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618292032326,"port_rcv_data":2271465408349,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.833","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.846","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.860","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.871","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.882","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.895","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.906","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.917","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.733962,"monotonic":2349953.661539216,"hcas":{"mlx5_0":{"port_xmit_data":111391022965645,"port_rcv_data":113096193603669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802588332127,"port_rcv_data":1495615814947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113399644080736,"port_rcv_data":112793684797522,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618292215846,"port_rcv_data":2271502762521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.832","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.845","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.858","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.871","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.882","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.892","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.903","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.913","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.7347717,"monotonic":2349954.662348772,"hcas":{"mlx5_0":{"port_xmit_data":111391035583721,"port_rcv_data":113097109499878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802588393275,"port_rcv_data":1495675608146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113400501352914,"port_rcv_data":112793780736083,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618292718542,"port_rcv_data":2271586104386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.833","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.846","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.860","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.872","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.883","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.894","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.904","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.914","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.73558,"monotonic":2349955.663156732,"hcas":{"mlx5_0":{"port_xmit_data":111391048472283,"port_rcv_data":113098020906583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802588733128,"port_rcv_data":1495704882024,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113401383577844,"port_rcv_data":112793807668976,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618292732632,"port_rcv_data":2271600464497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.817","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.831","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.856","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.867","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.878","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":142.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.889","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":143.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.898","gpu_util_pct":98.0,"memory_used_mib":80045.0,"power_w":140.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.7364094,"monotonic":2349956.663986088,"hcas":{"mlx5_0":{"port_xmit_data":111391060449715,"port_rcv_data":113098989862878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802589104105,"port_rcv_data":1495747388774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113402308888157,"port_rcv_data":112793861715621,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618293115110,"port_rcv_data":2271642971247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.873","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.887","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.899","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.912","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.923","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.934","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.944","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.954","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.737179,"monotonic":2349957.664756028,"hcas":{"mlx5_0":{"port_xmit_data":111391072212058,"port_rcv_data":113099947908185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802589307511,"port_rcv_data":1495786384686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113403227897300,"port_rcv_data":112793921111032,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618293483855,"port_rcv_data":2271690806967,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:52.838","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:52.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.864","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.875","gpu_util_pct":99.0,"memory_used_mib":79923.0,"power_w":143.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.886","gpu_util_pct":99.0,"memory_used_mib":79915.0,"power_w":135.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.897","gpu_util_pct":99.0,"memory_used_mib":79959.0,"power_w":139.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.907","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":145.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.917","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.7379825,"monotonic":2349958.665559433,"hcas":{"mlx5_0":{"port_xmit_data":111391085477705,"port_rcv_data":113100851605873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802589777998,"port_rcv_data":1495817171253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113404103331956,"port_rcv_data":112793947747763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618293664786,"port_rcv_data":2271704609954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.838","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.868","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.880","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.892","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.903","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.913","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.923","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.738783,"monotonic":2349959.666359988,"hcas":{"mlx5_0":{"port_xmit_data":111391098458664,"port_rcv_data":113101789677769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802590034260,"port_rcv_data":1495887048368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113404969734131,"port_rcv_data":112794038425300,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618294044257,"port_rcv_data":2271782627397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.820","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.864","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.876","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.888","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.899","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.910","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.7395966,"monotonic":2349960.667173423,"hcas":{"mlx5_0":{"port_xmit_data":111391111393470,"port_rcv_data":113102726558259,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802590256328,"port_rcv_data":1495955466255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113405840333223,"port_rcv_data":112794119619616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618294266078,"port_rcv_data":2271851045348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.821","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.834","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.848","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.860","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.872","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.895","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.905","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.7404156,"monotonic":2349961.66799222,"hcas":{"mlx5_0":{"port_xmit_data":111391124137834,"port_rcv_data":113103650686456,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802590317429,"port_rcv_data":1496014437632,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113406703726618,"port_rcv_data":112794191255422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618294326776,"port_rcv_data":2271910016725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:56.838","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.864","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.876","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.887","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.898","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.910","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.920","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.740446,"monotonic":2349962.668023069,"hcas":{"mlx5_0":{"port_xmit_data":111391137109160,"port_rcv_data":113104564006608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802590610968,"port_rcv_data":1496060101059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113407573617441,"port_rcv_data":112794251366652,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618294672767,"port_rcv_data":2271957404238,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.858","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.871","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.883","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.895","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.906","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.917","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.928","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.938","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.7412338,"monotonic":2349963.668810674,"hcas":{"mlx5_0":{"port_xmit_data":111391150755690,"port_rcv_data":113105369323430,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802590855093,"port_rcv_data":1496100232172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113408339080762,"port_rcv_data":112794311956618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618295024028,"port_rcv_data":2272004584702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:58.840","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":142.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.866","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.878","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.889","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.899","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.909","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.919","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.7420409,"monotonic":2349964.669617566,"hcas":{"mlx5_0":{"port_xmit_data":111391164455785,"port_rcv_data":113106179745523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802591205006,"port_rcv_data":1496132490450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113409117911272,"port_rcv_data":112794348783515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618295206060,"port_rcv_data":2272028071040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.841","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.867","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.879","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.890","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.901","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.910","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.920","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.742849,"monotonic":2349965.670425648,"hcas":{"mlx5_0":{"port_xmit_data":111391177621446,"port_rcv_data":113107073181498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802591548098,"port_rcv_data":1496150121195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113409993302911,"port_rcv_data":112794379407606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618295565117,"port_rcv_data":2272045909242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.823","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.837","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.862","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.873","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.904","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.7436757,"monotonic":2349966.671252554,"hcas":{"mlx5_0":{"port_xmit_data":111391190733427,"port_rcv_data":113107986037761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802591892927,"port_rcv_data":1496192430181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113410863313180,"port_rcv_data":112794428555037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618295752683,"port_rcv_data":2272082418295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:01.881","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.895","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.907","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.919","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.929","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.940","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.949","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.959","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.7444491,"monotonic":2349967.672025917,"hcas":{"mlx5_0":{"port_xmit_data":111391202651199,"port_rcv_data":113108936324746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802592106920,"port_rcv_data":1496255841060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113411751135858,"port_rcv_data":112794509273789,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618296119041,"port_rcv_data":2272151420737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.844","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.858","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.870","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.882","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.892","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.903","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.912","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.922","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.745259,"monotonic":2349968.672835815,"hcas":{"mlx5_0":{"port_xmit_data":111391214358467,"port_rcv_data":113109893851000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802592312233,"port_rcv_data":1496297022688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113412668564778,"port_rcv_data":112794567982435,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618296466070,"port_rcv_data":2272198640165,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.845","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.884","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.896","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.907","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.918","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.928","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.7460647,"monotonic":2349969.673641518,"hcas":{"mlx5_0":{"port_xmit_data":111391226050641,"port_rcv_data":113110846857917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802592481307,"port_rcv_data":1496332596716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113413585089228,"port_rcv_data":112794609006224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618296497334,"port_rcv_data":2272228176393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.827","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.840","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.853","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.865","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.876","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.906","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.7468913,"monotonic":2349970.674468029,"hcas":{"mlx5_0":{"port_xmit_data":111391237671719,"port_rcv_data":113111800211682,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802592511392,"port_rcv_data":1496362140346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113414508498913,"port_rcv_data":112794650095141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618296527791,"port_rcv_data":2272257720087,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.844","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.857","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.870","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.881","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.891","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.902","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.912","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.921","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.7473273,"monotonic":2349971.674903672,"hcas":{"mlx5_0":{"port_xmit_data":111391249719721,"port_rcv_data":113112694747894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802592541974,"port_rcv_data":1496391683976,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113415373181904,"port_rcv_data":112794691622222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618296558032,"port_rcv_data":2272287263717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:06.849","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.875","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.886","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.897","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.908","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.918","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.927","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.748143,"monotonic":2349972.675719581,"hcas":{"mlx5_0":{"port_xmit_data":111391262805467,"port_rcv_data":113113594071424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802592724703,"port_rcv_data":1496410236778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113416254039602,"port_rcv_data":112794731850554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618296923134,"port_rcv_data":2272314589338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.866","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.879","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.892","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.904","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.915","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.936","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.945","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.748943,"monotonic":2349973.67651976,"hcas":{"mlx5_0":{"port_xmit_data":111391275797241,"port_rcv_data":113114529119638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802592937689,"port_rcv_data":1496473319486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113417126514695,"port_rcv_data":112794816804294,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618297299149,"port_rcv_data":2272386832618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.848","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.875","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.886","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.897","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.908","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.918","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.927","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.7497616,"monotonic":2349974.67733826,"hcas":{"mlx5_0":{"port_xmit_data":111391289140842,"port_rcv_data":113115462008183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802593443764,"port_rcv_data":1496529432120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113418004754192,"port_rcv_data":112794858984524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618297329560,"port_rcv_data":2272416169749,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.830","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.844","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.857","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.868","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.879","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.890","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.900","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.909","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.7505999,"monotonic":2349975.678176674,"hcas":{"mlx5_0":{"port_xmit_data":111391301267850,"port_rcv_data":113116445923646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802593635483,"port_rcv_data":1496567730339,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113418951830914,"port_rcv_data":112794926489229,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618297820724,"port_rcv_data":2272471775336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.865","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.879","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.891","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.904","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.915","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.936","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.946","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.7514021,"monotonic":2349976.678978706,"hcas":{"mlx5_0":{"port_xmit_data":111391313440460,"port_rcv_data":113117437857655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802593981319,"port_rcv_data":1496614870157,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113419895629159,"port_rcv_data":112794976970141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618298022952,"port_rcv_data":2272510333932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.852","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.865","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.879","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.891","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.903","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.913","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.923","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.933","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.7522168,"monotonic":2349977.67979344,"hcas":{"mlx5_0":{"port_xmit_data":111391325001387,"port_rcv_data":113118384749055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802594011389,"port_rcv_data":1496644297904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113420813712099,"port_rcv_data":112795017906884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618298053146,"port_rcv_data":2272539761711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.852","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.866","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.879","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.893","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.905","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.917","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.928","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.938","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.7524457,"monotonic":2349978.680022375,"hcas":{"mlx5_0":{"port_xmit_data":111391336684328,"port_rcv_data":113119321708684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802594228544,"port_rcv_data":1496668178700,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113421725123921,"port_rcv_data":112795053191474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618298246509,"port_rcv_data":2272563700249,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:13.852","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.866","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.879","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.891","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.902","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.912","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.922","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.932","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.753253,"monotonic":2349979.680829415,"hcas":{"mlx5_0":{"port_xmit_data":111391348552962,"port_rcv_data":113120297062585,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802594473987,"port_rcv_data":1496715203148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113422653985476,"port_rcv_data":112795117335038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618298603427,"port_rcv_data":2272616313521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.852","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.866","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.879","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.890","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.901","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.912","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.922","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.931","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.7540631,"monotonic":2349980.681639724,"hcas":{"mlx5_0":{"port_xmit_data":111391360794555,"port_rcv_data":113121297005139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802594957788,"port_rcv_data":1496777256836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113423592102885,"port_rcv_data":112795176789055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618298803145,"port_rcv_data":2272664064941,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.853","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.880","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.891","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.902","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.912","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.922","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.932","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.7548819,"monotonic":2349981.682458619,"hcas":{"mlx5_0":{"port_xmit_data":111391372754893,"port_rcv_data":113122287954917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802595165752,"port_rcv_data":1496830199206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113424530374842,"port_rcv_data":112795250135457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618299159257,"port_rcv_data":2272725661947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:16.835","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.861","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.874","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.885","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.896","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.906","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.916","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.755722,"monotonic":2349982.683298661,"hcas":{"mlx5_0":{"port_xmit_data":111391384711767,"port_rcv_data":113123293639645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802595389665,"port_rcv_data":1496897973685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113425469790365,"port_rcv_data":112795338530803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618299532063,"port_rcv_data":2272802355675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.910","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.923","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.936","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.947","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.958","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.968","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.978","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.987","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.756451,"monotonic":2349983.684027537,"hcas":{"mlx5_0":{"port_xmit_data":111391396451901,"port_rcv_data":113124342455181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802595620599,"port_rcv_data":1496979370448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113426436582591,"port_rcv_data":112795422471340,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618299604324,"port_rcv_data":2272874891549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.872","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.885","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.898","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.910","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.921","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.932","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.942","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.952","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.7572443,"monotonic":2349984.684821153,"hcas":{"mlx5_0":{"port_xmit_data":111391407459627,"port_rcv_data":113125365734365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802595829183,"port_rcv_data":1497017008655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113427421677801,"port_rcv_data":112795461906998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618299633103,"port_rcv_data":2272903611125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.837","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.867","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.901","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.931","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.957","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.969","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.982","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.7579994,"monotonic":2349985.685576292,"hcas":{"mlx5_0":{"port_xmit_data":111391418321260,"port_rcv_data":113126363393370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802596029257,"port_rcv_data":1497040287320,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113428396508962,"port_rcv_data":112795513353863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618300139209,"port_rcv_data":2272944520767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.856","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.869","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.882","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.895","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.906","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.916","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.926","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":146.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.935","gpu_util_pct":99.0,"memory_used_mib":80045.0,"power_w":145.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.7588112,"monotonic":2349986.686387968,"hcas":{"mlx5_0":{"port_xmit_data":111391430055992,"port_rcv_data":113127300234816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802596337877,"port_rcv_data":1497058125152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113429317642010,"port_rcv_data":112795533881329,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618300314778,"port_rcv_data":2272953647425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:21.873","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.899","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.910","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.921","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.931","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.941","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.950","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.7595892,"monotonic":2349987.687165941,"hcas":{"mlx5_0":{"port_xmit_data":111391441831943,"port_rcv_data":113128314029036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802596708234,"port_rcv_data":1497115022465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113430274894665,"port_rcv_data":112795593165634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618300524477,"port_rcv_data":2273001866852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.860","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.886","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.898","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.910","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.921","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.931","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.940","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.760399,"monotonic":2349988.687975992,"hcas":{"mlx5_0":{"port_xmit_data":111391452761669,"port_rcv_data":113129328851207,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802596886639,"port_rcv_data":1497147753305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113431257639460,"port_rcv_data":112795654303449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618301023004,"port_rcv_data":2273052324452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.862","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.875","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.900","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.911","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.932","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.942","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.7604434,"monotonic":2349989.688020142,"hcas":{"mlx5_0":{"port_xmit_data":111391463898586,"port_rcv_data":113130347979794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802597217316,"port_rcv_data":1497179839745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113432242378396,"port_rcv_data":112795688202898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618301195224,"port_rcv_data":2273075525732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.860","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.886","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.897","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.908","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.919","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.929","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.939","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.7612534,"monotonic":2349990.688830127,"hcas":{"mlx5_0":{"port_xmit_data":111391475029337,"port_rcv_data":113131374979885,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802597399891,"port_rcv_data":1497203041025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113433248887155,"port_rcv_data":112795731166364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618301528738,"port_rcv_data":2273107568612,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.860","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.886","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.898","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.909","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.919","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.929","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.939","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.7620633,"monotonic":2349991.689640136,"hcas":{"mlx5_0":{"port_xmit_data":111391486421369,"port_rcv_data":113132417855788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802597785252,"port_rcv_data":1497238346912,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113434256524955,"port_rcv_data":112795765306299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618301699935,"port_rcv_data":2273130770108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.861","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.874","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.887","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.899","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.910","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.920","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.930","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.940","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.7628815,"monotonic":2349992.690458186,"hcas":{"mlx5_0":{"port_xmit_data":111391497454563,"port_rcv_data":113133449708902,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802597972957,"port_rcv_data":1497276855569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113435248796222,"port_rcv_data":112795817718910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618301938341,"port_rcv_data":2273172542728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.859","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.872","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.885","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.897","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.909","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.919","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.929","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":141.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.939","gpu_util_pct":99.0,"memory_used_mib":80045.0,"power_w":143.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.7636943,"monotonic":2349993.691271257,"hcas":{"mlx5_0":{"port_xmit_data":111391507692885,"port_rcv_data":113134469746720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802597973530,"port_rcv_data":1497279198296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113436266381380,"port_rcv_data":112795827870856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618301938341,"port_rcv_data":2273172542728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.864","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.878","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.891","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.903","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.915","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.925","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.944","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.764444,"monotonic":2349994.692020696,"hcas":{"mlx5_0":{"port_xmit_data":111391517275804,"port_rcv_data":113135441111930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802598370222,"port_rcv_data":1497333353677,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113437185108031,"port_rcv_data":112795893423011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618302322973,"port_rcv_data":2273229039152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.864","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.877","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.890","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.901","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.912","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.932","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.941","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.7652583,"monotonic":2349995.692835169,"hcas":{"mlx5_0":{"port_xmit_data":111391527408978,"port_rcv_data":113136496862554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802598749507,"port_rcv_data":1497409065409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113438165453864,"port_rcv_data":112795978787959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618302706629,"port_rcv_data":2273304750948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.865","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.878","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.890","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.902","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.913","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.923","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.933","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.943","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.7660701,"monotonic":2349996.693646638,"hcas":{"mlx5_0":{"port_xmit_data":111391537587202,"port_rcv_data":113137509158558,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599039868,"port_rcv_data":1497426187561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113439160994879,"port_rcv_data":112796005719740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303007949,"port_rcv_data":2273321873686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.863","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.877","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.890","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.901","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.912","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.923","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.933","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.942","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.7668898,"monotonic":2349997.694466254,"hcas":{"mlx5_0":{"port_xmit_data":111391547843962,"port_rcv_data":113138529990655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599333020,"port_rcv_data":1497443309713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113440164834941,"port_rcv_data":112796032739354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303302682,"port_rcv_data":2273338996424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.881","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.894","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.907","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.918","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.930","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.940","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.950","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.960","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.7676852,"monotonic":2349998.695261891,"hcas":{"mlx5_0":{"port_xmit_data":111391556239868,"port_rcv_data":113139393230690,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599333020,"port_rcv_data":1497443309713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113441025653441,"port_rcv_data":112796040964917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303302682,"port_rcv_data":2273338996424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.885","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.898","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.912","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.925","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.936","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.947","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.957","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.967","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.7684739,"monotonic":2349999.696050626,"hcas":{"mlx5_0":{"port_xmit_data":111391564396873,"port_rcv_data":113140273712274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599333020,"port_rcv_data":1497443309713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113441908378603,"port_rcv_data":112796048986726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303302682,"port_rcv_data":2273338996424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.848","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":134.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.874","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.886","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.896","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":122.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.906","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.916","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.926","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.769307,"monotonic":2350000.696883577,"hcas":{"mlx5_0":{"port_xmit_data":111391572448157,"port_rcv_data":113141139061896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599333020,"port_rcv_data":1497443309713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113442771596285,"port_rcv_data":112796056846609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303302682,"port_rcv_data":2273338996424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.850","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":135.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.863","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.876","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.888","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.898","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":124.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.908","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.918","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.927","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.7695544,"monotonic":2350001.697131009,"hcas":{"mlx5_0":{"port_xmit_data":111391582278594,"port_rcv_data":113142146525551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599342506,"port_rcv_data":1497452882833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113443768958409,"port_rcv_data":112796076144785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303312168,"port_rcv_data":2273348569544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.867","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.881","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.906","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.917","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.927","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.937","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.947","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.770374,"monotonic":2350002.697950947,"hcas":{"mlx5_0":{"port_xmit_data":111391592069530,"port_rcv_data":113143145028104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599356518,"port_rcv_data":1497467242513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113444755220449,"port_rcv_data":112796100220688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303326180,"port_rcv_data":2273362929224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.885","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.898","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.911","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.923","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.934","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.945","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.955","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.965","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.7711751,"monotonic":2350003.698751703,"hcas":{"mlx5_0":{"port_xmit_data":111391602095989,"port_rcv_data":113144182817886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599384557,"port_rcv_data":1497495961873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113445764224499,"port_rcv_data":112796138859258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303354591,"port_rcv_data":2273391648584,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.851","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.865","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.877","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.889","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.900","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.910","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.920","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.930","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.772015,"monotonic":2350004.699591717,"hcas":{"mlx5_0":{"port_xmit_data":111391611402939,"port_rcv_data":113145125424655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599398693,"port_rcv_data":1497510321553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113446692501991,"port_rcv_data":112796162402683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303369223,"port_rcv_data":2273406008264,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.872","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.897","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.909","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.920","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.940","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.949","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.772833,"monotonic":2350005.700409645,"hcas":{"mlx5_0":{"port_xmit_data":111391620931984,"port_rcv_data":113146092106591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599412659,"port_rcv_data":1497524681233,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113447645051500,"port_rcv_data":112796186173824,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303383189,"port_rcv_data":2273420367944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.872","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.885","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.898","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.910","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.921","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.931","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.941","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.951","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.7731016,"monotonic":2350006.70067824,"hcas":{"mlx5_0":{"port_xmit_data":111391631945384,"port_rcv_data":113147288630336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599426655,"port_rcv_data":1497539040913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113448824601766,"port_rcv_data":112796211504797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303397201,"port_rcv_data":2273434727624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.872","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.899","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.911","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.922","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.933","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.943","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.953","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.7739146,"monotonic":2350007.701491423,"hcas":{"mlx5_0":{"port_xmit_data":111391643227752,"port_rcv_data":113148494369725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802599606812,"port_rcv_data":1497547882513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113450024325442,"port_rcv_data":112796238107212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618303666715,"port_rcv_data":2273450275064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.909","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.947","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.958","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.968","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.978","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.988","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.7746925,"monotonic":2350008.702269236,"hcas":{"mlx5_0":{"port_xmit_data":111391654821348,"port_rcv_data":113149811266884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600055909,"port_rcv_data":1497664735826,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113451224808333,"port_rcv_data":112796361937811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304038777,"port_rcv_data":2273562973091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.889","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.902","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.915","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.927","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.939","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.949","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.959","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.969","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.7754846,"monotonic":2350009.703061527,"hcas":{"mlx5_0":{"port_xmit_data":111391666229030,"port_rcv_data":113151024593227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600370869,"port_rcv_data":1497682366502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113452420948829,"port_rcv_data":112796390938625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304369671,"port_rcv_data":2273580918070,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.876","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.889","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.902","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.914","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.926","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.936","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.946","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.956","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.7762918,"monotonic":2350010.703868685,"hcas":{"mlx5_0":{"port_xmit_data":111391677677731,"port_rcv_data":113152243319637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600674266,"port_rcv_data":1497699944654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113453622592185,"port_rcv_data":112796416718098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304644904,"port_rcv_data":2273595547958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.845","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.886","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.898","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.909","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.920","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.929","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.7771275,"monotonic":2350011.704704096,"hcas":{"mlx5_0":{"port_xmit_data":111391688862905,"port_rcv_data":113153440923400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113454809868751,"port_rcv_data":112796436501228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.876","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.889","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.902","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.914","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.925","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.935","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":146.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.945","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":146.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.955","gpu_util_pct":98.0,"memory_used_mib":80045.0,"power_w":146.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.7779505,"monotonic":2350012.705527276,"hcas":{"mlx5_0":{"port_xmit_data":111391699526928,"port_rcv_data":113154692021836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113456060190088,"port_rcv_data":112796447154843,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.893","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.907","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.920","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.931","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.942","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.953","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.963","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.972","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.778745,"monotonic":2350013.706321384,"hcas":{"mlx5_0":{"port_xmit_data":111391710084513,"port_rcv_data":113155940589719,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113457311481910,"port_rcv_data":112796457734427,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.893","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.906","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.919","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.931","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":151.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.942","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.952","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.962","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.972","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.7795403,"monotonic":2350014.707117206,"hcas":{"mlx5_0":{"port_xmit_data":111391720663648,"port_rcv_data":113157192364652,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113458562616156,"port_rcv_data":112796468308253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.860","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.886","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.899","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.910","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.921","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.931","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.941","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.7803674,"monotonic":2350015.707944247,"hcas":{"mlx5_0":{"port_xmit_data":111391731156765,"port_rcv_data":113158432325995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113459801665446,"port_rcv_data":112796478795364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.912","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.950","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.961","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.971","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.981","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.991","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.7811484,"monotonic":2350016.708725254,"hcas":{"mlx5_0":{"port_xmit_data":111391741807871,"port_rcv_data":113159692676959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113461061945322,"port_rcv_data":112796489450562,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.861","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.875","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.899","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.910","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.920","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.930","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.939","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.7819858,"monotonic":2350017.709562384,"hcas":{"mlx5_0":{"port_xmit_data":111391752469500,"port_rcv_data":113160953979187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113462322218375,"port_rcv_data":112796500099991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.880","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.893","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.914","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.941","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.972","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.001","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.014","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.027","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.7827213,"monotonic":2350018.710298069,"hcas":{"mlx5_0":{"port_xmit_data":111391763127332,"port_rcv_data":113162214362083,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113463582500278,"port_rcv_data":112796510755975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.880","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.893","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.906","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.919","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":151.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.930","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.941","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.951","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.961","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.783531,"monotonic":2350019.711107808,"hcas":{"mlx5_0":{"port_xmit_data":111391773621383,"port_rcv_data":113163453405509,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113464821551452,"port_rcv_data":112796521245126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.921","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.934","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.947","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.960","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.971","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.982","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.992","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.001","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.7842689,"monotonic":2350020.711845502,"hcas":{"mlx5_0":{"port_xmit_data":111391784228664,"port_rcv_data":113164708707523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113466079106392,"port_rcv_data":112796531876494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.885","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.898","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.911","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.922","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.933","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.963","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.7850795,"monotonic":2350021.712656501,"hcas":{"mlx5_0":{"port_xmit_data":111391794864343,"port_rcv_data":113165966685056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113467337570064,"port_rcv_data":112796542509791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.884","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.897","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.910","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.921","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.932","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.962","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.7858999,"monotonic":2350022.713476504,"hcas":{"mlx5_0":{"port_xmit_data":111391805213581,"port_rcv_data":113167193515500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113468564437023,"port_rcv_data":112796552848375,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.884","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.897","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.910","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.921","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.932","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.942","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.952","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.962","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.7867122,"monotonic":2350023.714288827,"hcas":{"mlx5_0":{"port_xmit_data":111391815932387,"port_rcv_data":113168455130737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113469823323391,"port_rcv_data":112796563557209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.884","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.897","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.909","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.921","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.933","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.963","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.7875228,"monotonic":2350024.7150998,"hcas":{"mlx5_0":{"port_xmit_data":111391826511999,"port_rcv_data":113169706624082,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113471077378205,"port_rcv_data":112796574159126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.868","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.881","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.906","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.916","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.927","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.937","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.946","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.7883492,"monotonic":2350025.715926123,"hcas":{"mlx5_0":{"port_xmit_data":111391837209446,"port_rcv_data":113170971856224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113472342375087,"port_rcv_data":112796584852150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.905","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.918","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.931","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.943","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.955","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.965","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.985","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.78914,"monotonic":2350026.71671686,"hcas":{"mlx5_0":{"port_xmit_data":111391847822246,"port_rcv_data":113172230270773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113473600738743,"port_rcv_data":112796595466259,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.870","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.908","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.940","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.949","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.789975,"monotonic":2350027.717551779,"hcas":{"mlx5_0":{"port_xmit_data":111391858427642,"port_rcv_data":113173475058910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113474843238248,"port_rcv_data":112796606034922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.888","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.902","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.914","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.926","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.938","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.948","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.958","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.968","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.7907872,"monotonic":2350028.718364168,"hcas":{"mlx5_0":{"port_xmit_data":111391869063520,"port_rcv_data":113174732610182,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600823035,"port_rcv_data":1497708733730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113476103523990,"port_rcv_data":112796616690697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304798540,"port_rcv_data":2273604337327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.907","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":157.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.920","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.933","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.945","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.957","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.968","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.978","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.988","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.7915795,"monotonic":2350029.719156295,"hcas":{"mlx5_0":{"port_xmit_data":111391879886387,"port_rcv_data":113175926587873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600832459,"port_rcv_data":1497718306850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113477288048137,"port_rcv_data":112796637099862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304807964,"port_rcv_data":2273613910447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.873","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.899","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.911","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.922","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.932","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.942","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.952","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.792409,"monotonic":2350030.71998565,"hcas":{"mlx5_0":{"port_xmit_data":111391890809037,"port_rcv_data":113177125504150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600855771,"port_rcv_data":1497742239650,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113478462989049,"port_rcv_data":112796671948005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304831385,"port_rcv_data":2273637843247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.893","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.906","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.919","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.931","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.943","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.954","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.964","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.974","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.7932224,"monotonic":2350031.720799239,"hcas":{"mlx5_0":{"port_xmit_data":111391901773804,"port_rcv_data":113178334090474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600883764,"port_rcv_data":1497770959010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113479642920047,"port_rcv_data":112796711626330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304859548,"port_rcv_data":2273666562607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.896","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.909","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.922","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.933","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.944","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.955","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.965","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.975","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.7940373,"monotonic":2350032.721613757,"hcas":{"mlx5_0":{"port_xmit_data":111391912744621,"port_rcv_data":113179569803372,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600939905,"port_rcv_data":1497828397730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113480818626568,"port_rcv_data":112796779980460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304915596,"port_rcv_data":2273724001327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.929","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.942","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.955","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.967","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.977","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.987","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.997","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.007","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.7948194,"monotonic":2350033.722395888,"hcas":{"mlx5_0":{"port_xmit_data":111391923604428,"port_rcv_data":113180766474492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600967867,"port_rcv_data":1497857117090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113481986656492,"port_rcv_data":112796819549453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304943605,"port_rcv_data":2273752720687,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.909","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.947","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.958","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.969","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.979","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.989","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.7956166,"monotonic":2350034.723193337,"hcas":{"mlx5_0":{"port_xmit_data":111391934521026,"port_rcv_data":113181970169216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802600995906,"port_rcv_data":1497885836450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113483164221310,"port_rcv_data":112796859210150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304971660,"port_rcv_data":2273781440047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.895","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.908","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.921","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.933","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.944","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.955","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.965","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.975","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.7964413,"monotonic":2350035.724017609,"hcas":{"mlx5_0":{"port_xmit_data":111391945605644,"port_rcv_data":113183170548401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802601187145,"port_rcv_data":1497909037730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113484341744287,"port_rcv_data":112796884494594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618304985734,"port_rcv_data":2273795799727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.877","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.892","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.907","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.920","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.932","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.963","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.7972705,"monotonic":2350036.724847125,"hcas":{"mlx5_0":{"port_xmit_data":111391956481629,"port_rcv_data":113184339389729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802601187145,"port_rcv_data":1497909037730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113485508356836,"port_rcv_data":112796921877075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618305499869,"port_rcv_data":2273822324527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.897","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.910","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.923","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.936","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.949","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.961","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.972","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.983","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.7980797,"monotonic":2350037.725656439,"hcas":{"mlx5_0":{"port_xmit_data":111391968217686,"port_rcv_data":113185549848248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802601635064,"port_rcv_data":1497979091993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113486651490707,"port_rcv_data":112796987946998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618305715102,"port_rcv_data":2273877200468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.913","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.927","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.951","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.962","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.974","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.984","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.994","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.7988784,"monotonic":2350038.726455002,"hcas":{"mlx5_0":{"port_xmit_data":111391979999535,"port_rcv_data":113186685793015,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802601914793,"port_rcv_data":1498014340073,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113487750335029,"port_rcv_data":112797041071559,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618306058721,"port_rcv_data":2273918816468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.913","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.927","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.951","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.962","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.972","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.983","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.993","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.7996714,"monotonic":2350039.727247978,"hcas":{"mlx5_0":{"port_xmit_data":111391991924621,"port_rcv_data":113187696718708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802602086641,"port_rcv_data":1498037541353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113488740295904,"port_rcv_data":112797084927159,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618306395397,"port_rcv_data":2273950859348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.880","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":159.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.894","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.907","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.919","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":150.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.930","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.941","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.951","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.961","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.8005,"monotonic":2350040.728076773,"hcas":{"mlx5_0":{"port_xmit_data":111392004780885,"port_rcv_data":113188671263779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802602483426,"port_rcv_data":1498073464898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113489679348028,"port_rcv_data":112797119493305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618306539020,"port_rcv_data":2273972980591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.918","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.932","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.944","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.956","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.967","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.978","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.988","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.998","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.8012912,"monotonic":2350041.72886808,"hcas":{"mlx5_0":{"port_xmit_data":111392017434964,"port_rcv_data":113189583699386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802602689839,"port_rcv_data":1498099466297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113490566278800,"port_rcv_data":112797154051069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618306681806,"port_rcv_data":2273995102264,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.882","gpu_util_pct":97.0,"memory_used_mib":79957.0,"power_w":151.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.896","gpu_util_pct":97.0,"memory_used_mib":79959.0,"power_w":146.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.909","gpu_util_pct":97.0,"memory_used_mib":79943.0,"power_w":143.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.921","gpu_util_pct":98.0,"memory_used_mib":79923.0,"power_w":141.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.932","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.963","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.8021264,"monotonic":2350042.729702994,"hcas":{"mlx5_0":{"port_xmit_data":111392030083998,"port_rcv_data":113190513749327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802602717863,"port_rcv_data":1498128185657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113491467030936,"port_rcv_data":112797195413401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618306709768,"port_rcv_data":2274023821624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.901","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.915","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.927","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.939","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.950","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.960","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.970","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.979","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.8029406,"monotonic":2350043.730517282,"hcas":{"mlx5_0":{"port_xmit_data":111392042539055,"port_rcv_data":113191404272617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802602733131,"port_rcv_data":1498130736017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113492355609728,"port_rcv_data":112797207895086,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618306709768,"port_rcv_data":2274023821624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.901","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.915","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.928","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.939","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.950","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.961","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.970","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.980","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.8037558,"monotonic":2350044.731332434,"hcas":{"mlx5_0":{"port_xmit_data":111392055268147,"port_rcv_data":113192357146155,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802602971180,"port_rcv_data":1498191991341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113493245696022,"port_rcv_data":112797301827930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618307261165,"port_rcv_data":2274105308184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.904","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.917","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.930","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.942","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.954","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.965","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.976","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.986","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.804567,"monotonic":2350045.732143744,"hcas":{"mlx5_0":{"port_xmit_data":111392068298996,"port_rcv_data":113193294978414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802603316148,"port_rcv_data":1498234192187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113494141564340,"port_rcv_data":112797347927066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618307447769,"port_rcv_data":2274138666130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.904","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.918","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.930","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.942","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.954","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.964","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.974","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.983","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.805386,"monotonic":2350046.732962607,"hcas":{"mlx5_0":{"port_xmit_data":111392081832506,"port_rcv_data":113194118199260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802603542200,"port_rcv_data":1498310654427,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113494890864036,"port_rcv_data":112797428900078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618307515318,"port_rcv_data":2274206285726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.925","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.939","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.952","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.964","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.975","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.996","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.006","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.8061833,"monotonic":2350047.733760226,"hcas":{"mlx5_0":{"port_xmit_data":111392095043449,"port_rcv_data":113194909324081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802603595349,"port_rcv_data":1498363915387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113495628278089,"port_rcv_data":112797495311204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618307568902,"port_rcv_data":2274259545514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.904","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.918","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.931","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.943","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.954","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.964","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.974","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.984","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.8070025,"monotonic":2350048.734579373,"hcas":{"mlx5_0":{"port_xmit_data":111392108067670,"port_rcv_data":113195607294075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802603793145,"port_rcv_data":1498415835892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113496274373678,"port_rcv_data":112797559899049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618307767968,"port_rcv_data":2274311259148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.941","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.954","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.967","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.979","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.990","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.019","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.8077865,"monotonic":2350049.735363157,"hcas":{"mlx5_0":{"port_xmit_data":111392120742750,"port_rcv_data":113196218016314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802604001108,"port_rcv_data":1498477362956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113496823329443,"port_rcv_data":112797642904990,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618308140557,"port_rcv_data":2274381835212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.908","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.921","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.947","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.959","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.970","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.980","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.989","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.8086038,"monotonic":2350050.736180242,"hcas":{"mlx5_0":{"port_xmit_data":111392133470543,"port_rcv_data":113196835329532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802604400016,"port_rcv_data":1498552434827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113497366345973,"port_rcv_data":112797721635889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618308354832,"port_rcv_data":2274448066126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.926","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.941","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.965","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.992","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.017","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.056","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.068","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.8093288,"monotonic":2350051.736905651,"hcas":{"mlx5_0":{"port_xmit_data":111392146105472,"port_rcv_data":113197470563706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802604616461,"port_rcv_data":1498604405315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113497949090779,"port_rcv_data":112797785950384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618308584619,"port_rcv_data":2274500036614,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.909","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.947","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.959","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.969","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.980","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.990","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.8101435,"monotonic":2350052.737720175,"hcas":{"mlx5_0":{"port_xmit_data":111392158592637,"port_rcv_data":113198054356303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802604830423,"port_rcv_data":1498646771053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113498492099028,"port_rcv_data":112797856642378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618309061229,"port_rcv_data":2274558426312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.909","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.923","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.947","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.958","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.968","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.978","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.987","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.8109486,"monotonic":2350053.738525341,"hcas":{"mlx5_0":{"port_xmit_data":111392171365311,"port_rcv_data":113198646894983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802605386284,"port_rcv_data":1498701965365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113499029950730,"port_rcv_data":112797899451555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618309152338,"port_rcv_data":2274588755064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.909","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":139.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.47,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.948","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.959","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.969","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.980","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.990","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.8117576,"monotonic":2350054.739334447,"hcas":{"mlx5_0":{"port_xmit_data":111392183594744,"port_rcv_data":113199221409868,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802605425018,"port_rcv_data":1498739274593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113499567645284,"port_rcv_data":112797975423986,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618309669883,"port_rcv_data":2274652587300,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.913","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":142.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.951","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.962","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.974","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.984","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.994","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.8124456,"monotonic":2350055.740022405,"hcas":{"mlx5_0":{"port_xmit_data":111392196400609,"port_rcv_data":113199825848735,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802605995031,"port_rcv_data":1498804949977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113500105291380,"port_rcv_data":112798035514550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618309887348,"port_rcv_data":2274700578312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.893","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.907","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.920","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.932","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.943","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.954","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.964","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.974","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.8132792,"monotonic":2350056.740855936,"hcas":{"mlx5_0":{"port_xmit_data":111392208788539,"port_rcv_data":113200426104627,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802606244937,"port_rcv_data":1498871581117,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113500639383067,"port_rcv_data":112798123242339,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618310318031,"port_rcv_data":2274776050540,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.893","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.907","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.920","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.931","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.942","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.952","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.962","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.972","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.8141117,"monotonic":2350057.741688508,"hcas":{"mlx5_0":{"port_xmit_data":111392221283789,"port_rcv_data":113201059938539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802606510612,"port_rcv_data":1498966655221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113501178809372,"port_rcv_data":112798236699085,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618310730182,"port_rcv_data":2274877402270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.912","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.950","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.961","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.973","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.982","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.992","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.81493,"monotonic":2350058.742506608,"hcas":{"mlx5_0":{"port_xmit_data":111392234059319,"port_rcv_data":113201687226223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802607043905,"port_rcv_data":1499054954424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113501718042671,"port_rcv_data":112798322178922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618310955614,"port_rcv_data":2274950681089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.931","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.944","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.03,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.957","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.969","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.979","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.989","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.999","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.009","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.815728,"monotonic":2350059.743304601,"hcas":{"mlx5_0":{"port_xmit_data":111392246439669,"port_rcv_data":113202288478616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802607272933,"port_rcv_data":1499121555285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113502254820864,"port_rcv_data":112798409856005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618311330512,"port_rcv_data":2275026089930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.916","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":141.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.929","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.47,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.942","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.954","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.964","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.975","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.985","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.995","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.8161685,"monotonic":2350060.743745071,"hcas":{"mlx5_0":{"port_xmit_data":111392259101744,"port_rcv_data":113202900631038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802607725241,"port_rcv_data":1499196026794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113502792369584,"port_rcv_data":112798485709275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618311611109,"port_rcv_data":2275089650130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.917","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":138.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.943","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.955","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.966","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.977","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.987","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.997","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.816986,"monotonic":2350061.744562805,"hcas":{"mlx5_0":{"port_xmit_data":111392271800149,"port_rcv_data":113203515273768,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802608138378,"port_rcv_data":1499269112107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113503335000609,"port_rcv_data":112798564219208,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618311899688,"port_rcv_data":2275155897902,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.916","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":140.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.943","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.955","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.966","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.977","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.987","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.996","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.8178062,"monotonic":2350062.74538302,"hcas":{"mlx5_0":{"port_xmit_data":111392284907729,"port_rcv_data":113204275748100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802608360307,"port_rcv_data":1499335276213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113504030180924,"port_rcv_data":112798660921324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618312425556,"port_rcv_data":2275239744588,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.916","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.942","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.955","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.966","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.977","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.987","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.997","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.8186235,"monotonic":2350063.746200085,"hcas":{"mlx5_0":{"port_xmit_data":111392298606871,"port_rcv_data":113205085541514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802608865545,"port_rcv_data":1499405201331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113504768870509,"port_rcv_data":112798735218862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618312799432,"port_rcv_data":2275300827227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.917","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.943","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.956","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.967","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.977","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.987","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.997","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.8194442,"monotonic":2350064.747020825,"hcas":{"mlx5_0":{"port_xmit_data":111392312112402,"port_rcv_data":113205869174063,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802609232786,"port_rcv_data":1499451878121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113505507131303,"port_rcv_data":112798804249023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618313336956,"port_rcv_data":2275356617945,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.920","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.933","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.947","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.959","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.970","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.981","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.991","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.001","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.8202589,"monotonic":2350065.7478356,"hcas":{"mlx5_0":{"port_xmit_data":111392325600518,"port_rcv_data":113206676802010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802609605267,"port_rcv_data":1499524010873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113506242227709,"port_rcv_data":112798896798646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618313815875,"port_rcv_data":2275436076026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.921","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.935","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.99,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.947","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.960","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.971","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.982","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.992","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.002","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.821077,"monotonic":2350066.748653699,"hcas":{"mlx5_0":{"port_xmit_data":111392338716873,"port_rcv_data":113207610270927,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802610281206,"port_rcv_data":1499601812387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113507100469174,"port_rcv_data":112798961782963,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618314005192,"port_rcv_data":2275488597114,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.920","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.934","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.947","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.958","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.969","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.42,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.979","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.989","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.999","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.8218985,"monotonic":2350067.749475251,"hcas":{"mlx5_0":{"port_xmit_data":111392351519699,"port_rcv_data":113208543740838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802610672426,"port_rcv_data":1499649556265,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113507985192052,"port_rcv_data":112799030747498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618314573841,"port_rcv_data":2275545149806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.937","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.951","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.964","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.976","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.987","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.998","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.008","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.017","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.8226993,"monotonic":2350068.750275972,"hcas":{"mlx5_0":{"port_xmit_data":111392363767864,"port_rcv_data":113209617448239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802611113246,"port_rcv_data":1499769326533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113508940764310,"port_rcv_data":112799164953224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618315006570,"port_rcv_data":2275667637774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.920","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.933","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.946","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.958","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.969","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.980","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.990","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.000","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.8235188,"monotonic":2350069.751095476,"hcas":{"mlx5_0":{"port_xmit_data":111392376006327,"port_rcv_data":113210676199042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802611623894,"port_rcv_data":1499841712217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113509925861133,"port_rcv_data":112799237542664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618315351662,"port_rcv_data":2275728495388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.904","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.918","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.930","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.942","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.953","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.964","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.974","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.983","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.8243527,"monotonic":2350070.751929437,"hcas":{"mlx5_0":{"port_xmit_data":111392387716699,"port_rcv_data":113211656291827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802611772415,"port_rcv_data":1499850415741,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113510899047454,"port_rcv_data":112799284089598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618315952860,"port_rcv_data":2275763447594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.941","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.956","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.969","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.980","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.991","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.001","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.012","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.021","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.8244553,"monotonic":2350071.752031747,"hcas":{"mlx5_0":{"port_xmit_data":111392399899404,"port_rcv_data":113212663892720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802612380635,"port_rcv_data":1499885574448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113511872659203,"port_rcv_data":112799304401613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618316101505,"port_rcv_data":2275772151920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.924","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.938","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.950","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.962","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.973","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.983","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.993","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.003","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.8252745,"monotonic":2350072.75285091,"hcas":{"mlx5_0":{"port_xmit_data":111392411864677,"port_rcv_data":113213664313099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802612651156,"port_rcv_data":1499902418182,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113512854726370,"port_rcv_data":112799343229204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618316572163,"port_rcv_data":2275799298636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.926","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.940","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.965","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.976","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.996","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.006","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.8260887,"monotonic":2350073.753665434,"hcas":{"mlx5_0":{"port_xmit_data":111392423784549,"port_rcv_data":113214655117592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613000635,"port_rcv_data":1499921769541,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113513826477377,"port_rcv_data":112799372906955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618316886224,"port_rcv_data":2275817396949,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.925","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.938","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.951","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.963","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.974","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.985","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.995","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.005","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.8269043,"monotonic":2350074.754481007,"hcas":{"mlx5_0":{"port_xmit_data":111392435548747,"port_rcv_data":113215719412231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474113,"port_rcv_data":1499948915740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113514864438149,"port_rcv_data":112799402320542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317197061,"port_rcv_data":2275835495263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.908","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.921","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.934","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.946","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.957","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.967","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.977","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.987","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.8277388,"monotonic":2350075.75531548,"hcas":{"mlx5_0":{"port_xmit_data":111392446867064,"port_rcv_data":113216823889986,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113515970956331,"port_rcv_data":112799422730436,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.932","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.946","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.959","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.971","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.982","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.993","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.013","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.8284497,"monotonic":2350076.756026443,"hcas":{"mlx5_0":{"port_xmit_data":111392457730600,"port_rcv_data":113217960620978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113517108368930,"port_rcv_data":112799433613417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.965","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.978","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.991","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.003","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.014","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.024","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.034","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.043","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.829226,"monotonic":2350077.756802703,"hcas":{"mlx5_0":{"port_xmit_data":111392468462157,"port_rcv_data":113219119587529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113518266812450,"port_rcv_data":112799444362241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.928","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.941","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.954","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.966","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.977","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.988","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.998","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.008","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.8300369,"monotonic":2350078.757613754,"hcas":{"mlx5_0":{"port_xmit_data":111392478928205,"port_rcv_data":113220309242728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113519457017489,"port_rcv_data":112799454822762,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.945","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.958","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.971","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.983","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.994","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.005","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.015","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.024","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.830831,"monotonic":2350079.758407679,"hcas":{"mlx5_0":{"port_xmit_data":111392489347800,"port_rcv_data":113221541935121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113520689620195,"port_rcv_data":112799465255112,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.949","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.964","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.976","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.988","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":151.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.999","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.009","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.019","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.029","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.831622,"monotonic":2350080.759198551,"hcas":{"mlx5_0":{"port_xmit_data":111392499745970,"port_rcv_data":113222773172780,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113521918155441,"port_rcv_data":112799475649296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.912","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.938","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.949","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.960","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.970","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.980","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.989","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.89,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.8324466,"monotonic":2350081.76002332,"hcas":{"mlx5_0":{"port_xmit_data":111392510048065,"port_rcv_data":113223989539987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113523136899058,"port_rcv_data":112799485984629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.912","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.936","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.969","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.001","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.018","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.030","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.042","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.052","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.8332198,"monotonic":2350082.760796542,"hcas":{"mlx5_0":{"port_xmit_data":111392520409645,"port_rcv_data":113225215342835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113524363124881,"port_rcv_data":112799496360456,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.932","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.946","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.959","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.970","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.981","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.992","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.002","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.011","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.8339984,"monotonic":2350083.761575133,"hcas":{"mlx5_0":{"port_xmit_data":111392530778780,"port_rcv_data":113226442055683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113525589334818,"port_rcv_data":112799506739825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.949","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.962","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.987","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":152.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.999","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.009","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.025","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.050","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.8347232,"monotonic":2350084.762300081,"hcas":{"mlx5_0":{"port_xmit_data":111392541114824,"port_rcv_data":113227662950547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113526808088496,"port_rcv_data":112799517065961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.915","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.928","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.941","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.952","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.963","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.973","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.983","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.992","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.835555,"monotonic":2350085.763131788,"hcas":{"mlx5_0":{"port_xmit_data":111392551465360,"port_rcv_data":113228887969430,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113528035713344,"port_rcv_data":112799527459233,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.973","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.987","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.999","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.011","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.022","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.032","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.042","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.051","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.8363278,"monotonic":2350086.763904521,"hcas":{"mlx5_0":{"port_xmit_data":111392561894597,"port_rcv_data":113230120926846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113529268522029,"port_rcv_data":112799537902898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.937","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.950","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.962","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.974","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.985","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.996","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.015","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.8364542,"monotonic":2350087.764030749,"hcas":{"mlx5_0":{"port_xmit_data":111392572304914,"port_rcv_data":113231317160185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613474144,"port_rcv_data":1499948915809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113530464655801,"port_rcv_data":112799548335163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317353363,"port_rcv_data":2275844544096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.953","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.967","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.980","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.992","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.004","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.014","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.024","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.034","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.8371594,"monotonic":2350088.76473627,"hcas":{"mlx5_0":{"port_xmit_data":111392583541642,"port_rcv_data":113232464500660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613497813,"port_rcv_data":1499972848609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113531585909940,"port_rcv_data":112799583472075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317376721,"port_rcv_data":2275868476896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.939","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.952","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.965","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.977","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.988","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.998","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.008","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.018","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.8376756,"monotonic":2350089.765252122,"hcas":{"mlx5_0":{"port_xmit_data":111392594805819,"port_rcv_data":113233576019418,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613512011,"port_rcv_data":1499987208289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113532685606702,"port_rcv_data":112799609140801,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317390795,"port_rcv_data":2275882836576,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.936","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.949","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.962","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.974","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.985","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.996","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.015","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.8384867,"monotonic":2350090.766063568,"hcas":{"mlx5_0":{"port_xmit_data":111392606155275,"port_rcv_data":113234694463856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613525992,"port_rcv_data":1500001567969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113533790077253,"port_rcv_data":112799634864552,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317404792,"port_rcv_data":2275897196256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.937","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.951","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.963","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.975","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.985","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.996","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.015","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.64,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.8392982,"monotonic":2350091.766875191,"hcas":{"mlx5_0":{"port_xmit_data":111392617472743,"port_rcv_data":113235809874932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613539988,"port_rcv_data":1500015927649,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113534888583027,"port_rcv_data":112799660519546,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317418866,"port_rcv_data":2275911555936,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.958","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.972","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.984","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.996","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.007","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.018","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.028","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.037","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.840088,"monotonic":2350092.76766473,"hcas":{"mlx5_0":{"port_xmit_data":111392628753340,"port_rcv_data":113236921826741,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613553985,"port_rcv_data":1500030287329,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113535988104120,"port_rcv_data":112799686181702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317432785,"port_rcv_data":2275925915616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.941","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.955","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.968","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.980","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.991","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.002","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.012","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.022","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.8404567,"monotonic":2350093.768033233,"hcas":{"mlx5_0":{"port_xmit_data":111392640119479,"port_rcv_data":113238044968971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613567997,"port_rcv_data":1500044647009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113537096958086,"port_rcv_data":112799711925599,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317446797,"port_rcv_data":2275940275296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.940","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.953","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.966","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.978","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.989","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.019","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.841274,"monotonic":2350094.768850655,"hcas":{"mlx5_0":{"port_xmit_data":111392651559464,"port_rcv_data":113239111953827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802613742341,"port_rcv_data":1500053488609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113538155905606,"port_rcv_data":112799749743476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618317956251,"port_rcv_data":2275966800096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.940","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.954","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.967","gpu_util_pct":97.0,"memory_used_mib":79943.0,"power_w":150.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.978","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.989","gpu_util_pct":98.0,"memory_used_mib":79915.0,"power_w":142.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.009","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.019","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259210.842092,"monotonic":2350095.769668776,"hcas":{"mlx5_0":{"port_xmit_data":111392664518548,"port_rcv_data":113240091861714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802614235830,"port_rcv_data":1500143782283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113539046068961,"port_rcv_data":112799835985074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618318183946,"port_rcv_data":2276040588016,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.940","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.954","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.967","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.978","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.989","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.999","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.009","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.019","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.8429117,"monotonic":2350096.770488422,"hcas":{"mlx5_0":{"port_xmit_data":111392677038387,"port_rcv_data":113241010687188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802614312648,"port_rcv_data":1500174475742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113539934257229,"port_rcv_data":112799877906252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618318214543,"port_rcv_data":2276069986891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.940","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.953","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.966","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.978","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.989","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.020","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.8437326,"monotonic":2350097.77130909,"hcas":{"mlx5_0":{"port_xmit_data":111392689457406,"port_rcv_data":113241953098334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802614372757,"port_rcv_data":1500233273493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113540817443816,"port_rcv_data":112799949101461,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618318274559,"port_rcv_data":2276128900160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.941","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.955","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.967","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.978","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.989","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.999","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.009","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.019","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.8444529,"monotonic":2350098.772029613,"hcas":{"mlx5_0":{"port_xmit_data":111392701962199,"port_rcv_data":113242854501841,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802614546310,"port_rcv_data":1500256344933,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113541696314562,"port_rcv_data":112799975817571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618318288788,"port_rcv_data":2276143259840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.945","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.958","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.971","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.982","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.993","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.003","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.013","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.023","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.8452702,"monotonic":2350099.772846979,"hcas":{"mlx5_0":{"port_xmit_data":111392714552957,"port_rcv_data":113243764493884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802614711695,"port_rcv_data":1500279623667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113542583567377,"port_rcv_data":112800029334352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618318767939,"port_rcv_data":2276184376716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.944","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.958","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.970","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.981","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.993","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.003","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.013","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.023","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.846088,"monotonic":2350100.773664781,"hcas":{"mlx5_0":{"port_xmit_data":111392727273496,"port_rcv_data":113244679003349,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802615066862,"port_rcv_data":1500312028197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113543464603740,"port_rcv_data":112800064972018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618318939819,"port_rcv_data":2276207656036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.980","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.994","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.007","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":145.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.019","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.030","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.041","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.051","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.061","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.8468702,"monotonic":2350101.774446835,"hcas":{"mlx5_0":{"port_xmit_data":111392739181187,"port_rcv_data":113245664417877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802615267324,"port_rcv_data":1500330442917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113544433144104,"port_rcv_data":112800103981524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618319306828,"port_rcv_data":2276234912356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.948","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.962","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.974","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.986","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.997","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.007","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.017","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.026","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.8476875,"monotonic":2350102.775264216,"hcas":{"mlx5_0":{"port_xmit_data":111392751075256,"port_rcv_data":113246680578123,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802615552183,"port_rcv_data":1500370958711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113545409251475,"port_rcv_data":112800157671595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618319660848,"port_rcv_data":2276277045150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.949","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.963","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.987","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.998","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.008","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.018","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.028","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.848452,"monotonic":2350103.776028774,"hcas":{"mlx5_0":{"port_xmit_data":111392762946334,"port_rcv_data":113247743366933,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802615852094,"port_rcv_data":1500463101595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113546380554576,"port_rcv_data":112800250943898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618319743525,"port_rcv_data":2276358728410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.949","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.962","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.987","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.998","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.008","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.019","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.028","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.849268,"monotonic":2350104.776844744,"hcas":{"mlx5_0":{"port_xmit_data":111392774664726,"port_rcv_data":113248769141569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802615897060,"port_rcv_data":1500506449681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113547362730913,"port_rcv_data":112800305966903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618319787297,"port_rcv_data":2276402075196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.949","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.962","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.988","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.999","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.020","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.030","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.8496366,"monotonic":2350105.7772133,"hcas":{"mlx5_0":{"port_xmit_data":111392786451684,"port_rcv_data":113249766187244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802615940646,"port_rcv_data":1500549797767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113548314678697,"port_rcv_data":112800361038135,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618319830790,"port_rcv_data":2276445421982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.950","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.964","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.977","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.989","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.000","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.020","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.029","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.850447,"monotonic":2350106.778023637,"hcas":{"mlx5_0":{"port_xmit_data":111392799001443,"port_rcv_data":113250701705595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802615984883,"port_rcv_data":1500593128267,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113549208919717,"port_rcv_data":112800416941234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618319874391,"port_rcv_data":2276488751438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.931","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.944","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.957","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.971","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.985","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.001","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.015","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.029","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.8512552,"monotonic":2350107.778831834,"hcas":{"mlx5_0":{"port_xmit_data":111392811697390,"port_rcv_data":113251630949102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802616188940,"port_rcv_data":1500630940687,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113550100677334,"port_rcv_data":112800467288626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618320076434,"port_rcv_data":2276526562686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.953","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.966","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.979","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.991","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":139.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.002","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.27,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.013","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.022","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.033","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.8520582,"monotonic":2350108.779634923,"hcas":{"mlx5_0":{"port_xmit_data":111392824408176,"port_rcv_data":113252556924031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802616398314,"port_rcv_data":1500664548251,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113550991421340,"port_rcv_data":112800530971334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618320642416,"port_rcv_data":2276577852278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.953","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.966","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.979","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.991","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.002","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.013","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.032","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.8524454,"monotonic":2350109.780022192,"hcas":{"mlx5_0":{"port_xmit_data":111392837444544,"port_rcv_data":113253506379103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802616924183,"port_rcv_data":1500720795127,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113551887184388,"port_rcv_data":112800582115974,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618320841095,"port_rcv_data":2276616415735,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.934","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.948","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.960","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.971","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.982","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.992","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.002","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.011","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.853276,"monotonic":2350110.780852746,"hcas":{"mlx5_0":{"port_xmit_data":111392850279700,"port_rcv_data":113254555238361,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802617229347,"port_rcv_data":1500875857065,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113552779450316,"port_rcv_data":112800766905138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618321471418,"port_rcv_data":2276788744927,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.970","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.983","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.996","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.008","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":138.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.019","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.029","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.039","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.048","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.8540702,"monotonic":2350111.781647088,"hcas":{"mlx5_0":{"port_xmit_data":111392862801700,"port_rcv_data":113255505327328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802617705398,"port_rcv_data":1500926724205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113553680717982,"port_rcv_data":112800812593296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618321650614,"port_rcv_data":2276822344028,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.952","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":146.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.965","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.978","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.989","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.000","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.011","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.020","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.030","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.8548892,"monotonic":2350112.782465895,"hcas":{"mlx5_0":{"port_xmit_data":111392874790151,"port_rcv_data":113256536099868,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802617888391,"port_rcv_data":1500964285165,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113554672691202,"port_rcv_data":112800879636655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618322148257,"port_rcv_data":2276877588188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.992","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":145.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.005","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.018","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.030","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.041","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.051","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.061","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.070","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.8556628,"monotonic":2350113.78323951,"hcas":{"mlx5_0":{"port_xmit_data":111392886942010,"port_rcv_data":113257558081617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802618363296,"port_rcv_data":1501005246961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113555655660529,"port_rcv_data":112800914613673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618322318772,"port_rcv_data":2276900867077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.956","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.969","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.982","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.994","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.005","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.015","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.025","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.035","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.8564465,"monotonic":2350114.78402312,"hcas":{"mlx5_0":{"port_xmit_data":111392899929909,"port_rcv_data":113258544925043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802618533904,"port_rcv_data":1501028525695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113556618101149,"port_rcv_data":112800968506615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618322813269,"port_rcv_data":2276941983953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.939","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":149.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.952","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.971","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.997","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.029","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.060","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.073","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.084","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.8572,"monotonic":2350115.784776663,"hcas":{"mlx5_0":{"port_xmit_data":111392913547711,"port_rcv_data":113259556453013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802619008530,"port_rcv_data":1501069702751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113557591183268,"port_rcv_data":112801005025257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618322986512,"port_rcv_data":2276965324039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.957","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.971","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.984","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.996","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.007","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.017","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.027","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.037","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.8580098,"monotonic":2350116.785586639,"hcas":{"mlx5_0":{"port_xmit_data":111392926110888,"port_rcv_data":113260628033962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802619097655,"port_rcv_data":1501090710129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113558642177124,"port_rcv_data":112801058838455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618323500678,"port_rcv_data":2277006622998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.957","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.971","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.984","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.996","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.007","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.018","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.028","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.038","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.8588119,"monotonic":2350117.786388535,"hcas":{"mlx5_0":{"port_xmit_data":111392939189622,"port_rcv_data":113261712464012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802619688840,"port_rcv_data":1501119843077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113559695740170,"port_rcv_data":112801074589212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618323540172,"port_rcv_data":2277009922444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.957","gpu_util_pct":98.0,"memory_used_mib":79957.0,"power_w":152.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.970","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.983","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.994","gpu_util_pct":98.0,"memory_used_mib":79923.0,"power_w":144.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.005","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.015","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":146.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.025","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":147.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.035","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.859627,"monotonic":2350118.787203671,"hcas":{"mlx5_0":{"port_xmit_data":111392951438299,"port_rcv_data":113262903488854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802619815506,"port_rcv_data":1501207481074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113560801759608,"port_rcv_data":112801203026266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618324244446,"port_rcv_data":2277126273501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.965","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.978","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.991","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.003","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.014","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.035","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.045","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.8598652,"monotonic":2350119.787441719,"hcas":{"mlx5_0":{"port_xmit_data":111392964389143,"port_rcv_data":113264016272148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802620289465,"port_rcv_data":1501269429457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113561852596145,"port_rcv_data":112801254314837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618324283583,"port_rcv_data":2277165050004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.960","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":159.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.973","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.986","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.998","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.010","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.021","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.031","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.040","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.8604472,"monotonic":2350120.788023867,"hcas":{"mlx5_0":{"port_xmit_data":111392976967140,"port_rcv_data":113265085075226,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802620464925,"port_rcv_data":1501292578351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113562900992681,"port_rcv_data":112801307375577,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618324749839,"port_rcv_data":2277205777360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.978","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.991","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.016","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.027","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.037","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.047","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.057","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.8612432,"monotonic":2350121.788819955,"hcas":{"mlx5_0":{"port_xmit_data":111392990008634,"port_rcv_data":113266180571097,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802620935056,"port_rcv_data":1501333305121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113563952383851,"port_rcv_data":112801343065991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618324914030,"port_rcv_data":2277228926840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.960","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.973","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.986","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.998","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.009","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.020","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.030","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.040","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.8620596,"monotonic":2350122.789636053,"hcas":{"mlx5_0":{"port_xmit_data":111393002499342,"port_rcv_data":113267240988417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802620949300,"port_rcv_data":1501347664801,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113565000804651,"port_rcv_data":112801369956255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618324928290,"port_rcv_data":2277243286520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.977","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.991","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.015","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":152.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.027","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.037","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.047","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.057","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.8628545,"monotonic":2350123.790430933,"hcas":{"mlx5_0":{"port_xmit_data":111393015044170,"port_rcv_data":113268312269082,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802620963266,"port_rcv_data":1501362024481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113566057246632,"port_rcv_data":112801396887418,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618324942302,"port_rcv_data":2277257646200,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.961","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.974","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.987","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.999","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.011","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.022","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.032","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.042","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.8636637,"monotonic":2350124.79124035,"hcas":{"mlx5_0":{"port_xmit_data":111393027638819,"port_rcv_data":113269390395808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802620986671,"port_rcv_data":1501385957281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113567109260728,"port_rcv_data":112801433384982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618324965692,"port_rcv_data":2277281579000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.944","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.957","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.970","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.982","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.993","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.003","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.013","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.023","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.8644495,"monotonic":2350125.792026283,"hcas":{"mlx5_0":{"port_xmit_data":111393040367758,"port_rcv_data":113270455350634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802621163913,"port_rcv_data":1501394798881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113568168386804,"port_rcv_data":112801472528316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618325488429,"port_rcv_data":2277308103800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.964","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":158.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.978","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.990","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.002","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.013","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.033","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.043","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.8652601,"monotonic":2350126.792836713,"hcas":{"mlx5_0":{"port_xmit_data":111393053436405,"port_rcv_data":113271593665301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802621644382,"port_rcv_data":1501474038255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113569227187056,"port_rcv_data":112801548137031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618325707382,"port_rcv_data":2277371234114,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.964","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.977","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.990","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.002","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":153.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.013","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.034","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.043","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":151.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.866077,"monotonic":2350127.793653716,"hcas":{"mlx5_0":{"port_xmit_data":111393065947878,"port_rcv_data":113272668959739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802621732872,"port_rcv_data":1501505020903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113570272185699,"port_rcv_data":112801590064837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618325737421,"port_rcv_data":2277400640830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.964","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":158.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.978","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.990","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.002","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":151.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.013","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.033","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.042","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.8661406,"monotonic":2350128.793717374,"hcas":{"mlx5_0":{"port_xmit_data":111393078455950,"port_rcv_data":113273748378686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802621762616,"port_rcv_data":1501534557131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113571319757154,"port_rcv_data":112801632086889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618325768003,"port_rcv_data":2277430177058,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.964","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.978","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.991","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.003","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.014","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.034","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.044","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.8667245,"monotonic":2350129.794301169,"hcas":{"mlx5_0":{"port_xmit_data":111393090998671,"port_rcv_data":113274830768354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802621792965,"port_rcv_data":1501564093359,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113572374886678,"port_rcv_data":112801674205596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618325797902,"port_rcv_data":2277459713286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.965","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":159.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.978","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.991","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.003","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.015","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.036","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.046","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.8675354,"monotonic":2350130.795112104,"hcas":{"mlx5_0":{"port_xmit_data":111393103508308,"port_rcv_data":113275908472479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802621824430,"port_rcv_data":1501593636989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113573423694616,"port_rcv_data":112801716248993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618325828918,"port_rcv_data":2277489256980,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.968","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":159.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.982","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.995","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.007","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.018","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.028","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.038","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.048","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.868352,"monotonic":2350131.795928499,"hcas":{"mlx5_0":{"port_xmit_data":111393116262400,"port_rcv_data":113276997482188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802622043392,"port_rcv_data":1501627235659,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113574476967412,"port_rcv_data":112801771221355,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618326189200,"port_rcv_data":2277531697250,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.985","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.999","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.011","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.023","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.034","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.044","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.054","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.064","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.8684533,"monotonic":2350132.796030234,"hcas":{"mlx5_0":{"port_xmit_data":111393128953756,"port_rcv_data":113278083183903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802622228757,"port_rcv_data":1501660710263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113575530220132,"port_rcv_data":112801825943946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618326529146,"port_rcv_data":2277574012282,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.974","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.988","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.001","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.013","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":154.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.024","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.034","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.044","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.054","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.869259,"monotonic":2350133.79683561,"hcas":{"mlx5_0":{"port_xmit_data":111393141890599,"port_rcv_data":113279209372695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802622767273,"port_rcv_data":1501745573080,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113576573653253,"port_rcv_data":112801904523624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618326713921,"port_rcv_data":2277640143758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.968","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.981","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":156.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.994","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.006","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":150.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.018","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.028","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.038","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.048","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.870069,"monotonic":2350134.797645609,"hcas":{"mlx5_0":{"port_xmit_data":111393154432020,"port_rcv_data":113280291424436,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802622797390,"port_rcv_data":1501774776739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113577627072230,"port_rcv_data":112801974257800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618327247602,"port_rcv_data":2277697334443,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.969","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.983","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.996","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.007","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":156.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.018","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.029","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.039","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.048","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.870879,"monotonic":2350135.798455524,"hcas":{"mlx5_0":{"port_xmit_data":111393167439433,"port_rcv_data":113281399274066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802623295529,"port_rcv_data":1501830918961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113578678180315,"port_rcv_data":112802015981131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618327277625,"port_rcv_data":2277726538472,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.951","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.965","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.978","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.989","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":151.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.000","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.011","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.020","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.030","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.8717089,"monotonic":2350136.799285588,"hcas":{"mlx5_0":{"port_xmit_data":111393179416042,"port_rcv_data":113282545342097,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802623325537,"port_rcv_data":1501860122344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113579793370315,"port_rcv_data":112802057137746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618327308129,"port_rcv_data":2277755741887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.972","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.998","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.010","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.021","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.041","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.051","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":154.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.8724515,"monotonic":2350137.8000282,"hcas":{"mlx5_0":{"port_xmit_data":111393191558665,"port_rcv_data":113283733320130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802623533717,"port_rcv_data":1501883807647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113580957917551,"port_rcv_data":112802092803351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618327494439,"port_rcv_data":2277779427190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.009","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":158.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.035","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.048","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":152.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.059","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.069","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.079","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.089","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.8732245,"monotonic":2350138.800801221,"hcas":{"mlx5_0":{"port_xmit_data":111393203813128,"port_rcv_data":113285014402515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802623809168,"port_rcv_data":1501994736315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113582131311820,"port_rcv_data":112802224559236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618327945737,"port_rcv_data":2277899195838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.972","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":158.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.999","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.011","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":150.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.022","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.033","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.043","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.053","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":151.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.8740396,"monotonic":2350139.801616265,"hcas":{"mlx5_0":{"port_xmit_data":111393216378426,"port_rcv_data":113286180792379,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802624280042,"port_rcv_data":1502020873725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113583269238060,"port_rcv_data":112802245502945,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618328105217,"port_rcv_data":2277908037438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.973","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.987","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.999","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.012","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":150.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.023","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.033","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.044","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.053","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.8748477,"monotonic":2350140.802424649,"hcas":{"mlx5_0":{"port_xmit_data":111393228868927,"port_rcv_data":113287240938763,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802624295620,"port_rcv_data":1502030781849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113584321587752,"port_rcv_data":112802293875307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618328580075,"port_rcv_data":2277943978234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.973","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.998","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.010","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.021","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.041","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.051","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.8756616,"monotonic":2350141.803238285,"hcas":{"mlx5_0":{"port_xmit_data":111393241701870,"port_rcv_data":113288338140080,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802624635674,"port_rcv_data":1502077079292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113585373548629,"port_rcv_data":112802343884718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618328761843,"port_rcv_data":2277981487179,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.976","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.990","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.002","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.014","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.025","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.035","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.045","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.055","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.8764691,"monotonic":2350142.804045776,"hcas":{"mlx5_0":{"port_xmit_data":111393254229003,"port_rcv_data":113289474788040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802624966429,"port_rcv_data":1502109276804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113586478815676,"port_rcv_data":112802397418224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618329245505,"port_rcv_data":2278022786557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.977","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.990","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.015","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":152.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.026","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.036","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.047","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.056","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.8772812,"monotonic":2350143.804858126,"hcas":{"mlx5_0":{"port_xmit_data":111393266748904,"port_rcv_data":113290690900205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802625444557,"port_rcv_data":1502150782683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113587654055813,"port_rcv_data":112802433111882,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618329417400,"port_rcv_data":2278046402950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.977","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.990","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.015","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.026","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.036","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.046","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.056","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.8780937,"monotonic":2350144.805670581,"hcas":{"mlx5_0":{"port_xmit_data":111393278688595,"port_rcv_data":113291865607081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802625458476,"port_rcv_data":1502165142363,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113588814086220,"port_rcv_data":112802459439529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618329431412,"port_rcv_data":2278060762630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.976","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.990","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.015","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.026","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.037","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.046","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.056","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.8789048,"monotonic":2350145.80648155,"hcas":{"mlx5_0":{"port_xmit_data":111393290781615,"port_rcv_data":113293057058140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802625472519,"port_rcv_data":1502179502043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113589988670180,"port_rcv_data":112802485884328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618329445440,"port_rcv_data":2278075122310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.031","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.044","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.057","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.069","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.080","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.091","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.100","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.110","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.8796675,"monotonic":2350146.807243911,"hcas":{"mlx5_0":{"port_xmit_data":111393303052703,"port_rcv_data":113294246158526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802625656768,"port_rcv_data":1502188343643,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113591169513914,"port_rcv_data":112802524539068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618329953375,"port_rcv_data":2278101647110,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.997","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.011","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.024","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.036","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.047","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.058","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.068","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.078","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.8804605,"monotonic":2350147.808037188,"hcas":{"mlx5_0":{"port_xmit_data":111393315743504,"port_rcv_data":113295553425409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626317417,"port_rcv_data":1502321958234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113592345927104,"port_rcv_data":112802645993779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330225725,"port_rcv_data":2278211095014,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.980","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.004","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.028","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.058","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.086","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.099","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.110","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.126","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.8811972,"monotonic":2350148.808773903,"hcas":{"mlx5_0":{"port_xmit_data":111393327904403,"port_rcv_data":113296734329025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626399304,"port_rcv_data":1502324395490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113593525299948,"port_rcv_data":112802693712264,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834379,"port_rcv_data":2278246771204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.962","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.976","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.989","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.000","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.011","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.022","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.032","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.042","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.8820295,"monotonic":2350149.80960623,"hcas":{"mlx5_0":{"port_xmit_data":111393340367961,"port_rcv_data":113297931263011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113594695638967,"port_rcv_data":112802705757410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.963","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":158.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.976","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.989","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.001","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":151.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.012","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.032","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.042","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.8828592,"monotonic":2350150.810436033,"hcas":{"mlx5_0":{"port_xmit_data":111393352121272,"port_rcv_data":113299110233129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113595874722957,"port_rcv_data":112802717550645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.980","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.993","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.018","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":150.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.029","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.040","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.049","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.059","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.8836732,"monotonic":2350151.811250022,"hcas":{"mlx5_0":{"port_xmit_data":111393363562743,"port_rcv_data":113300346229385,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113597110258250,"port_rcv_data":112802729019870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.984","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.998","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.022","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.033","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.053","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.062","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.8844907,"monotonic":2350152.812067346,"hcas":{"mlx5_0":{"port_xmit_data":111393374988972,"port_rcv_data":113301575273340,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113598337248817,"port_rcv_data":112802740439992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.984","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.998","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.021","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.032","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.042","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.052","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.062","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.8853056,"monotonic":2350153.812882317,"hcas":{"mlx5_0":{"port_xmit_data":111393386442367,"port_rcv_data":113302801595619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113599565614457,"port_rcv_data":112802751949589,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.002","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.016","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.041","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.053","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.063","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.074","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.084","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.8860962,"monotonic":2350154.813672919,"hcas":{"mlx5_0":{"port_xmit_data":111393398464856,"port_rcv_data":113303973115717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113600735214149,"port_rcv_data":112802763979588,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.984","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":153.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.998","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.011","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.023","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.034","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.044","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.054","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.064","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.886906,"monotonic":2350155.814482762,"hcas":{"mlx5_0":{"port_xmit_data":111393410489130,"port_rcv_data":113305147471371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113601911899272,"port_rcv_data":112802776068572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.984","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.997","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.022","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":150.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.033","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.053","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.063","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.887719,"monotonic":2350156.81529567,"hcas":{"mlx5_0":{"port_xmit_data":111393422403853,"port_rcv_data":113306304094693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113603067260653,"port_rcv_data":112802787995961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.988","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.002","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.015","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.026","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.038","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.048","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.44,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.058","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.069","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.8885353,"monotonic":2350157.816111871,"hcas":{"mlx5_0":{"port_xmit_data":111393434474023,"port_rcv_data":113307481456439,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113604244285743,"port_rcv_data":112802800097746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.005","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.019","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.032","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.044","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.055","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.065","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.075","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.084","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.8893278,"monotonic":2350158.81690458,"hcas":{"mlx5_0":{"port_xmit_data":111393446555262,"port_rcv_data":113308660216390,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113605424610275,"port_rcv_data":112802812226121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.988","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":155.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.002","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.015","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.026","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.037","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.057","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.067","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.8901417,"monotonic":2350159.817718564,"hcas":{"mlx5_0":{"port_xmit_data":111393458519079,"port_rcv_data":113309827405481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113606592374222,"port_rcv_data":112802824242118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.971","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.984","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.996","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.008","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.020","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.030","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.040","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.050","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.8909743,"monotonic":2350160.818550889,"hcas":{"mlx5_0":{"port_xmit_data":111393470566613,"port_rcv_data":113311003397659,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113607768351321,"port_rcv_data":112802836329947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.972","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.998","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.010","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":151.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.021","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.032","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.042","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.051","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":151.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.8918116,"monotonic":2350161.819388429,"hcas":{"mlx5_0":{"port_xmit_data":111393482529928,"port_rcv_data":113312169055987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113608933652608,"port_rcv_data":112802848314444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.010","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":157.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.037","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.049","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":148.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.060","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.071","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.081","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.091","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":149.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.8926082,"monotonic":2350162.820184914,"hcas":{"mlx5_0":{"port_xmit_data":111393494458647,"port_rcv_data":113313331732471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113610096662149,"port_rcv_data":112802860292784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.974","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.988","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.000","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.013","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.024","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.85,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.035","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.045","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.055","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.892661,"monotonic":2350163.820237589,"hcas":{"mlx5_0":{"port_xmit_data":111393506575349,"port_rcv_data":113314447567725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113611212726040,"port_rcv_data":112802872445715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.009","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":156.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.036","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.047","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":149.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.058","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.068","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.078","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.088","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":150.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.8934577,"monotonic":2350164.821034319,"hcas":{"mlx5_0":{"port_xmit_data":111393519069128,"port_rcv_data":113315493280332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113612256946788,"port_rcv_data":112802884956687,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.029","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.056","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.067","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":147.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.078","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.089","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.099","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.109","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.8942366,"monotonic":2350165.821813331,"hcas":{"mlx5_0":{"port_xmit_data":111393532496610,"port_rcv_data":113316451806600,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113613216760241,"port_rcv_data":112802898441571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.975","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.988","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.001","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.012","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.023","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.034","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.044","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.053","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.895066,"monotonic":2350166.822643132,"hcas":{"mlx5_0":{"port_xmit_data":111393545905281,"port_rcv_data":113317408674033,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113614173721886,"port_rcv_data":112802911881052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.975","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.989","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.002","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.014","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.026","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.037","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.047","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.057","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.8959022,"monotonic":2350167.823478709,"hcas":{"mlx5_0":{"port_xmit_data":111393559361736,"port_rcv_data":113318369530068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113615132511995,"port_rcv_data":112802925342548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.014","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.027","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.041","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.054","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":146.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.065","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.075","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.085","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.095","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.8966992,"monotonic":2350168.824275799,"hcas":{"mlx5_0":{"port_xmit_data":111393572742784,"port_rcv_data":113319324965761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113616090202079,"port_rcv_data":112802938810064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.997","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.011","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.024","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.035","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.046","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.057","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.067","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.076","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.8975215,"monotonic":2350169.825098218,"hcas":{"mlx5_0":{"port_xmit_data":111393586097815,"port_rcv_data":113320281138414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113617046732806,"port_rcv_data":112802952214482,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.997","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.035","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.045","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.056","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.066","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.076","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.8983405,"monotonic":2350170.825916877,"hcas":{"mlx5_0":{"port_xmit_data":111393599578449,"port_rcv_data":113321240772972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113618006110919,"port_rcv_data":112802965717504,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.996","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.009","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.035","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.046","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.056","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.066","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.076","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.899159,"monotonic":2350171.826735736,"hcas":{"mlx5_0":{"port_xmit_data":111393613020267,"port_rcv_data":113322200019535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113618963322993,"port_rcv_data":112802979160356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.996","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":154.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.035","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":145.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.046","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.056","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.067","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.076","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.8999796,"monotonic":2350172.827556119,"hcas":{"mlx5_0":{"port_xmit_data":111393626403867,"port_rcv_data":113323155888634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113619921539936,"port_rcv_data":112802992619065,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.017","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.044","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.057","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.068","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.078","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.088","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.098","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.900779,"monotonic":2350173.828355712,"hcas":{"mlx5_0":{"port_xmit_data":111393640590423,"port_rcv_data":113323972170171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113620737754036,"port_rcv_data":112803006842432,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.000","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.014","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.027","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.039","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":144.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.050","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.061","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.071","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.081","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.901597,"monotonic":2350174.829173537,"hcas":{"mlx5_0":{"port_xmit_data":111393654871980,"port_rcv_data":113324772977259,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113621538214052,"port_rcv_data":112803021142025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.984","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.997","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.023","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":142.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.034","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.045","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.055","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.065","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.9024289,"monotonic":2350175.830005518,"hcas":{"mlx5_0":{"port_xmit_data":111393669188914,"port_rcv_data":113325576937492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113622341565212,"port_rcv_data":112803035473559,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.001","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":151.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.014","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.027","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.039","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":143.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.049","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.060","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.070","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.080","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.9032505,"monotonic":2350176.830827006,"hcas":{"mlx5_0":{"port_xmit_data":111393683496838,"port_rcv_data":113326379780148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113623145600716,"port_rcv_data":112803049827804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.006","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":152.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.019","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.032","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.044","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":141.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.055","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.066","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.076","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.085","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.9040673,"monotonic":2350177.831643847,"hcas":{"mlx5_0":{"port_xmit_data":111393697763930,"port_rcv_data":113327179914468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113623943670302,"port_rcv_data":112803064078076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.007","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":150.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.021","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.033","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.045","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":140.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.056","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.066","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.076","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.085","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.9048803,"monotonic":2350178.832456952,"hcas":{"mlx5_0":{"port_xmit_data":111393711340608,"port_rcv_data":113327821418744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113624586876148,"port_rcv_data":112803077750061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.037","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":148.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.050","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.063","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.074","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.086","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.096","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.105","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.115","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.9056594,"monotonic":2350179.833236097,"hcas":{"mlx5_0":{"port_xmit_data":111393724740247,"port_rcv_data":113328411763923,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113625175731240,"port_rcv_data":112803091170474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.005","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":143.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.019","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.033","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.047","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.059","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.070","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.080","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.091","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.906455,"monotonic":2350180.834031815,"hcas":{"mlx5_0":{"port_xmit_data":111393738185336,"port_rcv_data":113329005176185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113625770189177,"port_rcv_data":112803104701257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.025","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":144.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.069","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.082","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":136.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.093","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.104","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.115","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.125","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.9072232,"monotonic":2350181.834799884,"hcas":{"mlx5_0":{"port_xmit_data":111393751569044,"port_rcv_data":113329594839772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113626360727059,"port_rcv_data":112803118162614,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.004","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":147.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.017","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.69,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.030","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.042","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.053","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.064","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.074","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.084","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.9080372,"monotonic":2350182.835613725,"hcas":{"mlx5_0":{"port_xmit_data":111393760688494,"port_rcv_data":113330007700890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113626772439982,"port_rcv_data":112803127323002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.009","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":137.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.036","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.048","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.059","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.075","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.100","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.132","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":126.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.908767,"monotonic":2350183.836343774,"hcas":{"mlx5_0":{"port_xmit_data":111393767661286,"port_rcv_data":113330332981247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113627097669613,"port_rcv_data":112803134371433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.009","gpu_util_pct":100.0,"memory_used_mib":79957.0,"power_w":136.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.036","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.048","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":128.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.059","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":118.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.070","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.081","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:39.090","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.9095836,"monotonic":2350184.837160302,"hcas":{"mlx5_0":{"port_xmit_data":111393768778186,"port_rcv_data":113330385035846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113627149431108,"port_rcv_data":112803135493510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.054","gpu_util_pct":0.0,"memory_used_mib":79957.0,"power_w":94.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.072","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":89.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.089","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":87.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.105","gpu_util_pct":0.0,"memory_used_mib":79923.0,"power_w":87.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.119","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":83.19,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.134","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":85.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:40.147","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":83.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.160","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":87.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.910278,"monotonic":2350185.837854619,"hcas":{"mlx5_0":{"port_xmit_data":111393768778186,"port_rcv_data":113330385035846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113627149431108,"port_rcv_data":112803135493510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.053","gpu_util_pct":0.0,"memory_used_mib":79957.0,"power_w":84.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.070","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":89.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.087","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":83.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.103","gpu_util_pct":0.0,"memory_used_mib":79923.0,"power_w":87.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.118","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":80.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.132","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":83.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.145","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.158","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":83.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.9110157,"monotonic":2350186.838592288,"hcas":{"mlx5_0":{"port_xmit_data":111393768778186,"port_rcv_data":113330385035846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113627149431108,"port_rcv_data":112803135493510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:42.051","gpu_util_pct":0.0,"memory_used_mib":79957.0,"power_w":83.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:42.069","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":85.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:42.086","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:42.102","gpu_util_pct":0.0,"memory_used_mib":79923.0,"power_w":82.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:42.116","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":80.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:42.130","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":83.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:42.144","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:42.157","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":83.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259302.9117534,"monotonic":2350187.839330216,"hcas":{"mlx5_0":{"port_xmit_data":111393768778186,"port_rcv_data":113330385035846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1802626858429,"port_rcv_data":1502351152031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":113627149431108,"port_rcv_data":112803135493510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2618330834519,"port_rcv_data":2278246771850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:43.053","gpu_util_pct":0.0,"memory_used_mib":79957.0,"power_w":83.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:43.071","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":85.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:43.089","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:43.104","gpu_util_pct":0.0,"memory_used_mib":79923.0,"power_w":82.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:43.118","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":80.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:43.132","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":83.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:43.145","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:43.159","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":83.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_6.jsonl

@@ -0,0 +1,459 @@
+{"wall_time":1788258845.1222863,"monotonic":2345649.55437226,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.221","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.246","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.263","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.278","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.292","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.306","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.319","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.332","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.123059,"monotonic":2345650.555145373,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.267","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.284","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.301","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.316","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.330","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.344","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.357","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.369","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.123795,"monotonic":2345651.555881283,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.246","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.259","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.271","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.283","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.294","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.306","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.317","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.329","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.123935,"monotonic":2345652.556021353,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.242","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.260","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.276","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.291","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.305","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.318","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.332","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.344","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.1246982,"monotonic":2345653.556784212,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.270","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.288","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.304","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.320","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.334","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.348","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.361","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.374","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.1254308,"monotonic":2345654.557517041,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.258","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.276","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.293","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.308","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.322","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.336","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.349","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.362","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.126176,"monotonic":2345655.558262194,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.275","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.293","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.310","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.326","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.340","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.354","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.367","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.380","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.1269143,"monotonic":2345656.559000328,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.247","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.265","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.281","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.296","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.310","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.324","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.337","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.350","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.1276915,"monotonic":2345657.559777088,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.275","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.293","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.310","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.325","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.340","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.353","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.367","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.380","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.1284413,"monotonic":2345658.560527487,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.251","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.269","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.286","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.301","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.316","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.330","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.343","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.356","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.1292155,"monotonic":2345659.561301338,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.273","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.292","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.310","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.327","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.343","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.358","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.373","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.387","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.129951,"monotonic":2345660.562036805,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.279","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.296","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.313","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.329","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.343","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.357","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.371","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.384","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.1306953,"monotonic":2345661.562781653,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.274","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.292","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.309","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.324","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.339","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.352","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.366","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.379","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.1314278,"monotonic":2345662.563514367,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.274","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.292","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.309","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.324","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.338","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.352","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.365","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.378","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.132167,"monotonic":2345663.564253637,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.278","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.296","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.312","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.327","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.342","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.356","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.369","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.382","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.1329048,"monotonic":2345664.564990933,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.278","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.296","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.313","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.328","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.343","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.357","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.370","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.383","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.1336415,"monotonic":2345665.565727811,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.274","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.292","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.309","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.325","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.339","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.353","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.366","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.379","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.1343846,"monotonic":2345666.566471107,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.258","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.276","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.293","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.308","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.323","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.336","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.350","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.363","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.1351454,"monotonic":2345667.567231959,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.254","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.272","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.288","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.303","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.318","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.332","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.345","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.358","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.135932,"monotonic":2345668.568018055,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.282","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.300","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.317","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.333","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.347","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.361","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.374","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.387","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.1366677,"monotonic":2345669.568753881,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.283","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.300","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.317","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.332","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.347","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.361","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.374","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.387","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.1374025,"monotonic":2345670.569488712,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.238","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.255","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.272","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.288","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.303","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.316","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.330","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.343","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.1381834,"monotonic":2345671.570269757,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.238","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.256","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.273","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.288","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.303","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.316","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.330","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.343","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.1389673,"monotonic":2345672.571053269,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.260","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.279","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.296","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.312","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.326","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.341","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.354","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.368","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.1397352,"monotonic":2345673.571821345,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.288","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.308","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.325","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.341","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.356","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.371","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.385","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.399","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.1399565,"monotonic":2345674.572042682,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.288","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.306","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.324","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.341","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.356","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.371","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.385","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.398","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.1406896,"monotonic":2345675.572775886,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.288","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.306","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.323","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.339","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.354","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.367","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.381","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.394","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.1414309,"monotonic":2345676.573516721,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.280","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.298","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.315","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.331","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.346","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.360","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.374","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.387","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.1421819,"monotonic":2345677.57426822,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.292","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.310","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.327","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.343","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.358","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.372","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.386","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.399","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.1429205,"monotonic":2345678.575006783,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.223","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.241","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.259","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.274","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.289","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.304","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.317","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.331","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.1437285,"monotonic":2345679.575814438,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.263","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.281","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.298","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.313","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.328","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.342","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.356","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.369","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.1444895,"monotonic":2345680.57657564,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.295","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.314","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.333","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.349","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.364","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.379","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.393","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.407","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.1452022,"monotonic":2345681.577288905,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.291","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.309","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.326","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.341","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.356","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.370","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.383","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:37.396","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.1459365,"monotonic":2345682.578022767,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.287","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.306","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.323","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.339","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.354","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.368","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.382","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.395","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.1466737,"monotonic":2345683.57875995,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.295","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.313","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.329","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.345","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.359","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.373","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.386","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.400","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.147407,"monotonic":2345684.579493402,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.291","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.310","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.326","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.342","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.357","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.371","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.384","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.398","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.147948,"monotonic":2345685.580034166,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.295","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.313","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.330","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.345","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.360","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.374","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.388","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:41.401","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.1486864,"monotonic":2345686.58077264,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.275","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.293","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.310","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.325","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.340","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.354","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.368","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.381","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.1494398,"monotonic":2345687.581526135,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.295","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.313","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.331","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.346","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.361","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.375","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.389","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.402","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.1501737,"monotonic":2345688.582260291,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.270","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.289","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.307","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.323","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.338","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.353","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.366","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.380","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.1506982,"monotonic":2345689.582784578,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.277","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.295","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.313","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.328","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.343","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.358","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.372","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.385","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.1514452,"monotonic":2345690.583531609,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.299","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.318","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.335","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.351","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.366","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.381","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.395","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.408","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.1521735,"monotonic":2345691.584260018,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.291","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.309","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.326","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.342","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.356","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.370","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.384","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:47.397","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.152911,"monotonic":2345692.584997043,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.254","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.272","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.289","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.305","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.320","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.334","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.347","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.360","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.153685,"monotonic":2345693.585771343,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.233","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.252","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.269","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.285","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.300","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.314","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.328","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.341","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.1544793,"monotonic":2345694.586565748,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.295","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.313","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.330","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.346","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.360","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.374","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.388","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.401","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.155214,"monotonic":2345695.587300443,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.259","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.276","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.292","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.307","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.321","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.335","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.348","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:51.361","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.1559403,"monotonic":2345696.588026525,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.259","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.276","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.293","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.308","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.322","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.336","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.349","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.362","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.1567252,"monotonic":2345697.588811585,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.278","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.296","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.313","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.328","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.342","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.356","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.369","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.382","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.1568272,"monotonic":2345698.588913457,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.279","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.297","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.314","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.329","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.344","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.357","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.371","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.384","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.1575847,"monotonic":2345699.589670762,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.304","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.321","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.338","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.354","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.368","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.382","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.395","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.408","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.1583164,"monotonic":2345700.590402517,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.303","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.321","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.338","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.353","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.368","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.382","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.396","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.409","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.1590471,"monotonic":2345701.591133572,"hcas":{"mlx5_0":{"port_xmit_data":98624620595100,"port_rcv_data":100198504513831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813194823,"port_rcv_data":2383456524939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764612854,"port_rcv_data":99161348626798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345853718,"port_rcv_data":2281254759193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.279","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.298","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.315","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.330","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.345","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.359","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.373","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.386","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.1598113,"monotonic":2345702.591897653,"hcas":{"mlx5_0":{"port_xmit_data":98624620604509,"port_rcv_data":100198514086951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813204201,"port_rcv_data":2383466098059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764622263,"port_rcv_data":99161358199918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345863142,"port_rcv_data":2281264332313,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.285","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.303","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.320","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.335","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.349","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.363","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.376","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.389","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.1599507,"monotonic":2345703.592037013,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.262","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.279","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.296","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.311","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.325","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.338","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.351","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.364","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.1607306,"monotonic":2345704.592817068,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.239","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.257","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.273","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.289","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.303","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.316","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.329","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.342","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.16154,"monotonic":2345705.593626556,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.283","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.300","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":76.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.317","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.332","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.346","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.360","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.373","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.386","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.1623003,"monotonic":2345706.594386829,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.278","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.296","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.312","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.328","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.342","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.356","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.369","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.382","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.1628973,"monotonic":2345707.594983665,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.286","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.303","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.319","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.334","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.348","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.362","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.375","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.388","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.163667,"monotonic":2345708.5957534,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.261","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.278","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.295","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.310","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.324","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.337","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.350","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.363","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.1644533,"monotonic":2345709.59653979,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.282","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.301","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.318","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.333","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.347","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.361","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.375","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.388","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.1652107,"monotonic":2345710.597297034,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.311","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.329","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.346","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.361","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.375","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.389","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.403","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.415","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.1659431,"monotonic":2345711.598029651,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.263","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.281","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.298","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.314","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.329","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.343","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.357","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.370","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.166729,"monotonic":2345712.598815306,"hcas":{"mlx5_0":{"port_xmit_data":98624620618536,"port_rcv_data":100198528446631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813218213,"port_rcv_data":2383480457739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764636244,"port_rcv_data":99161372559598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345877154,"port_rcv_data":2281278691993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.286","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.304","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.321","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.336","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.350","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.364","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.377","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.390","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.167491,"monotonic":2345713.59957728,"hcas":{"mlx5_0":{"port_xmit_data":98624620632533,"port_rcv_data":100198542806311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813232132,"port_rcv_data":2383494817419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764650225,"port_rcv_data":99161386919278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345891150,"port_rcv_data":2281293051673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.292","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.309","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.326","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.341","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.355","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.368","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.382","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.394","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.167941,"monotonic":2345714.600027284,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.311","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.330","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.347","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.363","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.378","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.392","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.406","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.419","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.1686733,"monotonic":2345715.600759511,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.314","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.333","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.350","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.365","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.380","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.394","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:11.408","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:11.421","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.1694148,"monotonic":2345716.601500726,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.314","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.332","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.349","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.364","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.379","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.393","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.406","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.419","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.1701493,"monotonic":2345717.602235591,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.265","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.284","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.300","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.316","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.330","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.344","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.358","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.371","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.1709282,"monotonic":2345718.603014613,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.314","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.332","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.348","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.363","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.378","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.391","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.405","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.417","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.1716666,"monotonic":2345719.603752844,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.294","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.312","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":76.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.328","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.343","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.358","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.372","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.385","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:15.398","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.1724267,"monotonic":2345720.604513042,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.319","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.338","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.355","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.372","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.387","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.402","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.416","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.430","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.1731536,"monotonic":2345721.605239773,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.314","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.332","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.349","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.365","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.380","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.394","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.407","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.420","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.173889,"monotonic":2345722.605975351,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.273","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.292","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.308","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.324","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.338","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.352","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.366","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.379","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.1746671,"monotonic":2345723.606753331,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.318","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.336","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.353","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.368","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.383","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:19.397","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:19.411","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:19.424","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.1754005,"monotonic":2345724.607486971,"hcas":{"mlx5_0":{"port_xmit_data":98624620646560,"port_rcv_data":100198557165991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813246144,"port_rcv_data":2383509177099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764664237,"port_rcv_data":99161401278958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345905038,"port_rcv_data":2281307411353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.277","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.295","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.312","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.327","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.341","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.354","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.368","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.380","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.1761777,"monotonic":2345725.608264056,"hcas":{"mlx5_0":{"port_xmit_data":98624620660572,"port_rcv_data":100198571525671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813260156,"port_rcv_data":2383523536779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764678249,"port_rcv_data":99161415638638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345919035,"port_rcv_data":2281321771033,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.320","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.337","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.354","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":75.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.370","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.384","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.398","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:21.411","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:21.424","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.1769032,"monotonic":2345726.608989999,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.278","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.296","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.312","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.327","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.341","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:22.354","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.367","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.380","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.177684,"monotonic":2345727.60977039,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.318","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.336","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.353","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.368","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.382","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.396","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.409","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.422","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.1784217,"monotonic":2345728.610508157,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.303","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.321","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.338","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.353","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.368","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.382","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:24.395","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:24.408","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.1791735,"monotonic":2345729.611259837,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.295","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.313","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.330","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.345","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.360","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.374","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.387","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.400","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.1799388,"monotonic":2345730.612025364,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.281","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.299","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.316","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.332","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.346","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.360","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.374","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.387","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.1803527,"monotonic":2345731.612438846,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.322","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.340","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.357","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.372","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.386","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.400","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.413","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:27.426","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.1811025,"monotonic":2345732.613188907,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.294","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.312","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.328","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.344","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.358","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.372","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.386","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.399","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.1818805,"monotonic":2345733.613966668,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.322","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.340","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.356","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.370","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.385","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.398","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:29.411","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:29.424","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.1826358,"monotonic":2345734.614721978,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.282","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.300","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":76.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.316","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.331","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.345","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.359","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.372","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.384","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.1834352,"monotonic":2345735.615521572,"hcas":{"mlx5_0":{"port_xmit_data":98624620674553,"port_rcv_data":100198585885351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813274137,"port_rcv_data":2383537896459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764692245,"port_rcv_data":99161429998318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345933000,"port_rcv_data":2281336130713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.263","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.280","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.297","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.312","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.327","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.340","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.354","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.366","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.18425,"monotonic":2345736.616336366,"hcas":{"mlx5_0":{"port_xmit_data":98624620688519,"port_rcv_data":100198600245031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813288133,"port_rcv_data":2383552256139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764706257,"port_rcv_data":99161444357998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345946997,"port_rcv_data":2281350490393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.331","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.349","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.366","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.381","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.395","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.409","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.422","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.435","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.1849847,"monotonic":2345737.617070949,"hcas":{"mlx5_0":{"port_xmit_data":98624620879882,"port_rcv_data":100198609086631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813475125,"port_rcv_data":2383561097739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635764706257,"port_rcv_data":99161444357998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643345946997,"port_rcv_data":2281350490393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.264","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.282","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.299","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.314","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.328","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.341","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.355","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.368","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.1857965,"monotonic":2345738.617882384,"hcas":{"mlx5_0":{"port_xmit_data":98624620983220,"port_rcv_data":100198705889576,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671813578309,"port_rcv_data":2383657929317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635765485256,"port_rcv_data":99161576508734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643346721020,"port_rcv_data":2281482710411,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.329","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.348","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.364","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.380","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.394","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.408","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:34.422","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:34.435","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.186545,"monotonic":2345739.618631153,"hcas":{"mlx5_0":{"port_xmit_data":98624621607622,"port_rcv_data":100198740983691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814198169,"port_rcv_data":2383693015124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635765485303,"port_rcv_data":99161576508965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643346721067,"port_rcv_data":2281482710627,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.328","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.346","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.363","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.378","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.393","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.406","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.420","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.433","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.187291,"monotonic":2345740.619377277,"hcas":{"mlx5_0":{"port_xmit_data":98624621607746,"port_rcv_data":100198740983967,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814198293,"port_rcv_data":2383693015400,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635766081495,"port_rcv_data":99161610786875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347322079,"port_rcv_data":2281516955241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.317","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.336","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.353","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.370","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.385","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.399","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":76.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:36.413","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:36.427","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.187939,"monotonic":2345741.620025183,"hcas":{"mlx5_0":{"port_xmit_data":98624622057448,"port_rcv_data":100198766691912,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814640136,"port_rcv_data":2383718698421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635766081634,"port_rcv_data":99161610787570,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347322219,"port_rcv_data":2281516955887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.335","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.353","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.370","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.386","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.401","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.415","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.429","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.442","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.1886697,"monotonic":2345742.620756036,"hcas":{"mlx5_0":{"port_xmit_data":98624626362835,"port_rcv_data":100198951460723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814640136,"port_rcv_data":2383718698421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99635951722378,"port_rcv_data":99161615074692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347322219,"port_rcv_data":2281516955887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.291","gpu_util_pct":100.0,"memory_used_mib":79971.0,"power_w":100.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.310","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":98.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.327","gpu_util_pct":100.0,"memory_used_mib":79917.0,"power_w":97.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.343","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":98.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.358","gpu_util_pct":100.0,"memory_used_mib":79889.0,"power_w":100.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.373","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":98.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.387","gpu_util_pct":100.0,"memory_used_mib":79917.0,"power_w":101.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.401","gpu_util_pct":99.0,"memory_used_mib":80019.0,"power_w":99.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.1894355,"monotonic":2345743.621521818,"hcas":{"mlx5_0":{"port_xmit_data":98624630254671,"port_rcv_data":100199120368527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814640136,"port_rcv_data":2383718698421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636119669506,"port_rcv_data":99161618900281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347322219,"port_rcv_data":2281516955887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.333","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":101.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.352","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":102.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.369","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":97.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.384","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":95.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.399","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":96.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.413","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":94.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.426","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":96.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.440","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":96.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.1901832,"monotonic":2345744.622269365,"hcas":{"mlx5_0":{"port_xmit_data":98624630254671,"port_rcv_data":100199120368527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814640136,"port_rcv_data":2383718698421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636119669506,"port_rcv_data":99161618900281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347322219,"port_rcv_data":2281516955887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.341","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":81.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.359","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.376","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.392","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.407","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":81.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.421","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:40.435","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":81.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.448","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.1905472,"monotonic":2345745.622633577,"hcas":{"mlx5_0":{"port_xmit_data":98624630278061,"port_rcv_data":100199144301327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814663557,"port_rcv_data":2383742631221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636119692926,"port_rcv_data":99161642833081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347345624,"port_rcv_data":2281540888687,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.328","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":81.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.346","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.363","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.379","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.393","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":81.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:41.407","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.421","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.434","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.191301,"monotonic":2345746.623387313,"hcas":{"mlx5_0":{"port_xmit_data":98624630301466,"port_rcv_data":100199168234127,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814686946,"port_rcv_data":2383766564021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636119716641,"port_rcv_data":99161666765881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347369308,"port_rcv_data":2281564821487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.337","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.45,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.354","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.371","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.386","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:42.400","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":81.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.413","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.427","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.439","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.191952,"monotonic":2345747.624038212,"hcas":{"mlx5_0":{"port_xmit_data":98624630338666,"port_rcv_data":100199206526607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814724363,"port_rcv_data":2383804856501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636119754074,"port_rcv_data":99161705058361,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347406663,"port_rcv_data":2281603113967,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.336","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.353","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.369","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.384","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.399","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.413","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.426","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.439","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.1926897,"monotonic":2345748.624775501,"hcas":{"mlx5_0":{"port_xmit_data":98624630397488,"port_rcv_data":100199266358607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814783278,"port_rcv_data":2383864688501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636119834069,"port_rcv_data":99161786429881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347486426,"port_rcv_data":2281684485487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.319","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.337","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.354","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.369","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.383","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.397","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.410","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.423","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.193444,"monotonic":2345749.625530376,"hcas":{"mlx5_0":{"port_xmit_data":98624630498718,"port_rcv_data":100199369269647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814884508,"port_rcv_data":2383967599541,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636119914608,"port_rcv_data":99161867801401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347566832,"port_rcv_data":2281765857007,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.344","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.363","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.381","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.396","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:45.412","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:45.426","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:45.440","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.453","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.1941788,"monotonic":2345750.626265081,"hcas":{"mlx5_0":{"port_xmit_data":98624630554596,"port_rcv_data":100199426708367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671814940572,"port_rcv_data":2384025038261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636120148704,"port_rcv_data":99161934081721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643347797986,"port_rcv_data":2281832137327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.275","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.293","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.311","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.327","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.342","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.357","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.371","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.384","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.1949832,"monotonic":2345751.627069374,"hcas":{"mlx5_0":{"port_xmit_data":98624630787871,"port_rcv_data":100199512519017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671815171785,"port_rcv_data":2384110897933,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636120726499,"port_rcv_data":99162037576871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643348372649,"port_rcv_data":2281935681371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.321","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.339","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.356","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.372","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.387","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.402","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.415","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.429","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.1957326,"monotonic":2345752.627818764,"hcas":{"mlx5_0":{"port_xmit_data":98624631359196,"port_rcv_data":100199610886161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671815735598,"port_rcv_data":2384209265077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636120987953,"port_rcv_data":99162118260815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643348629670,"port_rcv_data":2282016365315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.343","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.362","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.379","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.395","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:48.410","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.424","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.438","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.452","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.196459,"monotonic":2345753.628545325,"hcas":{"mlx5_0":{"port_xmit_data":98624631786759,"port_rcv_data":100199695618914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671816159240,"port_rcv_data":2384294010292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636121425176,"port_rcv_data":99162203006030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643349065313,"port_rcv_data":2282101098068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.299","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.318","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.335","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.351","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.364","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.376","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.389","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.401","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.1972523,"monotonic":2345754.629338478,"hcas":{"mlx5_0":{"port_xmit_data":98624632215350,"port_rcv_data":100199800432823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671816578112,"port_rcv_data":2384398889775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636121827044,"port_rcv_data":99162307819939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643349469300,"port_rcv_data":2282205977551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.344","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.362","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.380","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.395","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.410","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.425","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.439","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.452","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.1979842,"monotonic":2345755.630070333,"hcas":{"mlx5_0":{"port_xmit_data":98624632607314,"port_rcv_data":100199897976809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671816968945,"port_rcv_data":2384496396202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636122257603,"port_rcv_data":99162405386063,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643349899627,"port_rcv_data":2282303464826,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.316","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.334","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.352","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:51.368","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:51.383","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.397","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.411","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.425","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.198749,"monotonic":2345756.630835221,"hcas":{"mlx5_0":{"port_xmit_data":98624633000611,"port_rcv_data":100199990696709,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671817361715,"port_rcv_data":2384589154929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636122652900,"port_rcv_data":99162498086745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643350307680,"port_rcv_data":2282396243089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.324","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.342","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.359","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.375","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.390","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.404","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.418","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.432","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.1995044,"monotonic":2345757.631590658,"hcas":{"mlx5_0":{"port_xmit_data":98624633225578,"port_rcv_data":100200066894938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671817586635,"port_rcv_data":2384665472305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636123187944,"port_rcv_data":99162591949118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643350843267,"port_rcv_data":2282490265195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.320","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.339","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.356","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.372","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.387","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.401","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.415","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.429","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.2002618,"monotonic":2345758.632348137,"hcas":{"mlx5_0":{"port_xmit_data":98624633845454,"port_rcv_data":100200170578324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671818214292,"port_rcv_data":2384769213453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636123617156,"port_rcv_data":99162686841712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643351273764,"port_rcv_data":2282585114959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.298","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.317","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.334","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.350","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.365","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.379","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.393","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.406","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.2010586,"monotonic":2345759.6331447,"hcas":{"mlx5_0":{"port_xmit_data":98624634272758,"port_rcv_data":100200265979036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671818634730,"port_rcv_data":2384864671761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636124046654,"port_rcv_data":99162782242360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643351702689,"port_rcv_data":2282680574227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.344","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.362","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.379","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.394","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:55.409","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.422","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.436","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.448","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.2018151,"monotonic":2345760.633901317,"hcas":{"mlx5_0":{"port_xmit_data":98624634707176,"port_rcv_data":100200361237560,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671819066653,"port_rcv_data":2384959897943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636124631190,"port_rcv_data":99162885527209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643352281614,"port_rcv_data":2282783801634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.343","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.362","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.379","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.395","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.410","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.424","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.438","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.452","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.2025633,"monotonic":2345761.634649728,"hcas":{"mlx5_0":{"port_xmit_data":98624635418704,"port_rcv_data":100200465060700,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671819773763,"port_rcv_data":2385063714347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636124850065,"port_rcv_data":99162963645076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643352499544,"port_rcv_data":2282861934689,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.348","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.366","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.384","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.399","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.414","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":81.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.429","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.443","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.456","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.2032993,"monotonic":2345762.63538554,"hcas":{"mlx5_0":{"port_xmit_data":98624635501590,"port_rcv_data":100200545250951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671819856502,"port_rcv_data":2385144010518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636125591554,"port_rcv_data":99163079201695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643353241048,"port_rcv_data":2282977597740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.323","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.342","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.358","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.375","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:58.390","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":81.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:58.405","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:58.419","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:58.433","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.2040699,"monotonic":2345763.636156247,"hcas":{"mlx5_0":{"port_xmit_data":98624636250001,"port_rcv_data":100200648844756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671820602817,"port_rcv_data":2385247570617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636125817637,"port_rcv_data":99163156271660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643353468852,"port_rcv_data":2283054633103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.351","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":81.03,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.370","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.387","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.404","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.419","gpu_util_pct":78.0,"memory_used_mib":79891.0,"power_w":82.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.434","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.448","gpu_util_pct":77.0,"memory_used_mib":79919.0,"power_w":80.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.462","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.2048128,"monotonic":2345764.636899379,"hcas":{"mlx5_0":{"port_xmit_data":98624642669025,"port_rcv_data":100200997600122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671821032787,"port_rcv_data":2385341666109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636381920084,"port_rcv_data":99163265053379,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643354069678,"port_rcv_data":2283157571155,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.305","gpu_util_pct":99.0,"memory_used_mib":79973.0,"power_w":106.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.324","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":103.48,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.342","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":104.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.358","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":104.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.373","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":108.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.388","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":105.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.403","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":119.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.417","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":105.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.2055764,"monotonic":2345765.63766284,"hcas":{"mlx5_0":{"port_xmit_data":98624650637119,"port_rcv_data":100201425749002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671821833408,"port_rcv_data":2385459388525,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99636691828869,"port_rcv_data":99163372215447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643354483280,"port_rcv_data":2283257612587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.306","gpu_util_pct":99.0,"memory_used_mib":79973.0,"power_w":127.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.325","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":120.03,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.343","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":119.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.359","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":123.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.374","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":123.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.388","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":119.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:01.403","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.01,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.417","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":121.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.2063432,"monotonic":2345766.638429572,"hcas":{"mlx5_0":{"port_xmit_data":98624658004303,"port_rcv_data":100201805131017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671822087872,"port_rcv_data":2385530667309,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99637000955227,"port_rcv_data":99163477002603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643355240812,"port_rcv_data":2283355417343,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.306","gpu_util_pct":99.0,"memory_used_mib":79973.0,"power_w":128.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.322","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":123.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.337","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":120.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.351","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":122.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.364","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":123.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.376","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":121.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.387","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.397","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":122.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.2071302,"monotonic":2345767.63921654,"hcas":{"mlx5_0":{"port_xmit_data":98624665562042,"port_rcv_data":100202202150636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671822674330,"port_rcv_data":2385624669221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99637306252353,"port_rcv_data":99163577906393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643355826138,"port_rcv_data":2283449420619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.325","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":126.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.338","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":119.19,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.351","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":122.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.363","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":123.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.374","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":124.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.385","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":117.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.395","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.405","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":121.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.2079546,"monotonic":2345768.640041096,"hcas":{"mlx5_0":{"port_xmit_data":98624673474036,"port_rcv_data":100202592565980,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671823466705,"port_rcv_data":2385704526011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99637615621093,"port_rcv_data":99163638329044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643356106642,"port_rcv_data":2283503013237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.307","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":128.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.321","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":121.61,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.334","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":120.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.346","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":121.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.357","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":121.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.367","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.377","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.386","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":121.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.2087686,"monotonic":2345769.640854852,"hcas":{"mlx5_0":{"port_xmit_data":98624683393714,"port_rcv_data":100203069617505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671823703840,"port_rcv_data":2385779330183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99638019608079,"port_rcv_data":99163749125293,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643356820246,"port_rcv_data":2283604078545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.307","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":127.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.321","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.22,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.334","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":122.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.347","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":121.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.358","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":122.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:05.369","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":121.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.379","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":126.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.388","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":125.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.2095773,"monotonic":2345770.641663667,"hcas":{"mlx5_0":{"port_xmit_data":98624695653068,"port_rcv_data":100203585308310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671824172591,"port_rcv_data":2385805855190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99638508937077,"port_rcv_data":99163786952240,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643357276690,"port_rcv_data":2283630188936,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.307","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":131.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.321","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.334","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.347","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":125.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.358","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":126.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.369","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":123.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.379","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":126.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.389","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":125.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.2103903,"monotonic":2345771.642476467,"hcas":{"mlx5_0":{"port_xmit_data":98624707940322,"port_rcv_data":100204106314886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671824776021,"port_rcv_data":2385840668527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99638995347998,"port_rcv_data":99163816116445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643357582273,"port_rcv_data":2283647734861,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.325","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":130.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.338","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":122.93,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.351","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.363","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":127.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.374","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":124.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.385","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":123.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.394","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":127.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.404","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":126.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.2111847,"monotonic":2345772.643271168,"hcas":{"mlx5_0":{"port_xmit_data":98624720112524,"port_rcv_data":100204615539194,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671825093213,"port_rcv_data":2385858351727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99639487508259,"port_rcv_data":99163854429838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643358054961,"port_rcv_data":2283674259661,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.291","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":132.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.305","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.318","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.329","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":124.8,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:08.340","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":127.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.351","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.361","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":126.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.371","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":124.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.2120137,"monotonic":2345773.644100167,"hcas":{"mlx5_0":{"port_xmit_data":98624732415102,"port_rcv_data":100205176802005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671825564878,"port_rcv_data":2385884876527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99640022770894,"port_rcv_data":99163892717863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643358523712,"port_rcv_data":2283700784461,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.328","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":132.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.342","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":126.72,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.355","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":124.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.367","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":126.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.378","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":124.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.388","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":125.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.398","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":130.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.408","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":128.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.2128108,"monotonic":2345774.644896849,"hcas":{"mlx5_0":{"port_xmit_data":98624745235147,"port_rcv_data":100205850081099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671826050803,"port_rcv_data":2385911401534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99640669845656,"port_rcv_data":99163927907057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643358911925,"port_rcv_data":2283723717384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.293","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":134.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.307","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":125.81,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.320","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":127.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.332","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":132.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.343","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":126.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.353","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":127.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.363","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":134.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.372","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":128.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.2136402,"monotonic":2345775.645726674,"hcas":{"mlx5_0":{"port_xmit_data":98624757858350,"port_rcv_data":100206512242231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671826362896,"port_rcv_data":2385930189654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99641312243512,"port_rcv_data":99163974218812,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643359454378,"port_rcv_data":2283757886377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.311","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":134.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.325","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":126.63,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.338","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":131.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.351","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":127.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.362","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":127.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.372","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":131.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:11.383","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":130.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.393","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":125.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.2144518,"monotonic":2345776.646538151,"hcas":{"mlx5_0":{"port_xmit_data":98624770808695,"port_rcv_data":100207193964707,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671826978711,"port_rcv_data":2385966108244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99641958569376,"port_rcv_data":99164001808296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643359749808,"port_rcv_data":2283773176865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.313","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":134.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.326","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":126.78,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.339","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":129.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.351","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":129.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:12.362","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":129.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:12.374","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":126.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.384","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":130.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.393","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":130.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.215262,"monotonic":2345777.647348233,"hcas":{"mlx5_0":{"port_xmit_data":98624783236407,"port_rcv_data":100207901944360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827440735,"port_rcv_data":2385992425759,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99642641265297,"port_rcv_data":99164040225434,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360203137,"port_rcv_data":2283799702130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.333","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":136.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.346","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":129.77,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.360","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":125.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.372","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.383","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":128.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.394","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":130.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.404","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.414","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":128.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.2160518,"monotonic":2345778.648138101,"hcas":{"mlx5_0":{"port_xmit_data":98624794985461,"port_rcv_data":100208683702462,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99643413309691,"port_rcv_data":99164053086333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.315","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":137.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.329","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":128.03,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.341","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":128.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.353","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.364","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":130.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.375","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":132.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.385","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.394","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":130.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.2168682,"monotonic":2345779.648954559,"hcas":{"mlx5_0":{"port_xmit_data":98624806696661,"port_rcv_data":100209464557989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99644194041885,"port_rcv_data":99164064640712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.333","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":134.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.347","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.71,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.360","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.372","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.383","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":131.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:15.393","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":126.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:15.404","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":129.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.413","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":133.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.2176604,"monotonic":2345780.64974701,"hcas":{"mlx5_0":{"port_xmit_data":98624818325859,"port_rcv_data":100210240412885,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99644969528293,"port_rcv_data":99164076116032,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.299","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":139.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.312","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":130.88,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.325","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.337","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.349","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":134.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.361","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":129.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.372","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":134.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.382","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":130.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.2184808,"monotonic":2345781.650567377,"hcas":{"mlx5_0":{"port_xmit_data":98624830081541,"port_rcv_data":100211024168591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99645753255224,"port_rcv_data":99164087716999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.299","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":135.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.312","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":130.2,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.325","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.338","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.349","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":131.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.360","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":127.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.370","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.379","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":130.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.2193072,"monotonic":2345782.651393858,"hcas":{"mlx5_0":{"port_xmit_data":98624841074024,"port_rcv_data":100211786110593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99646515344475,"port_rcv_data":99164098559075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.319","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.333","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.19,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.346","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.358","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.369","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.380","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.390","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.399","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.2201161,"monotonic":2345783.652202344,"hcas":{"mlx5_0":{"port_xmit_data":98624851736031,"port_rcv_data":100212618585651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99647347260970,"port_rcv_data":99164109034512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.300","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.314","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.09,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.327","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.339","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.350","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.361","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.371","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.381","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.220887,"monotonic":2345784.652973601,"hcas":{"mlx5_0":{"port_xmit_data":98624862215652,"port_rcv_data":100213460356659,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99648186987603,"port_rcv_data":99164119316529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.301","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.315","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.38,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.328","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":135.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.339","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.350","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:20.361","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.371","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.380","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.2217133,"monotonic":2345785.653799728,"hcas":{"mlx5_0":{"port_xmit_data":98624872331783,"port_rcv_data":100214368825357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99649097513834,"port_rcv_data":99164129270590,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.302","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.315","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.94,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.328","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.341","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.353","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.364","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.374","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.384","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.222053,"monotonic":2345786.654139224,"hcas":{"mlx5_0":{"port_xmit_data":98624882365728,"port_rcv_data":100215268258620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99649996823677,"port_rcv_data":99164139100622,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.319","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.332","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":127.85,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.346","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":134.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.359","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.370","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:22.381","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":134.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.392","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":139.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.401","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.222783,"monotonic":2345787.654869153,"hcas":{"mlx5_0":{"port_xmit_data":98624890529040,"port_rcv_data":100216142237311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99650869852025,"port_rcv_data":99164147071384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.319","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.332","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.345","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.358","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.370","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.381","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.391","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.401","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.223594,"monotonic":2345788.655680607,"hcas":{"mlx5_0":{"port_xmit_data":98624898861623,"port_rcv_data":100217055695070,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99651783963989,"port_rcv_data":99164155241364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.322","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.336","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.350","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.362","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:24.373","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.383","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.394","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.403","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.2239316,"monotonic":2345789.656017955,"hcas":{"mlx5_0":{"port_xmit_data":98624906952138,"port_rcv_data":100217940925339,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99652667094728,"port_rcv_data":99164163145768,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.324","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.337","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.41,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.350","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":122.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.361","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.372","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.383","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.393","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.403","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.224741,"monotonic":2345790.656827305,"hcas":{"mlx5_0":{"port_xmit_data":98624915089245,"port_rcv_data":100218832753964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99653558712476,"port_rcv_data":99164171125023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.324","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.337","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.77,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.350","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.362","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:26.373","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.383","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.393","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.402","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.2252698,"monotonic":2345791.657356369,"hcas":{"mlx5_0":{"port_xmit_data":98624923214041,"port_rcv_data":100219723384918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99654449146257,"port_rcv_data":99164179072230,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.303","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.316","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.39,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.329","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.341","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.352","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.363","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.373","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.383","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.2259166,"monotonic":2345792.658003064,"hcas":{"mlx5_0":{"port_xmit_data":98624932237162,"port_rcv_data":100220685179657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827602431,"port_rcv_data":2386002822295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99655410225426,"port_rcv_data":99164187922270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360214219,"port_rcv_data":2283801051717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.324","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.337","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.52,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:28.350","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:28.361","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:28.372","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:28.382","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.392","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.402","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.2265582,"monotonic":2345793.658644609,"hcas":{"mlx5_0":{"port_xmit_data":98624941122154,"port_rcv_data":100221599786805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827625960,"port_rcv_data":2386026755095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99656303324515,"port_rcv_data":99164220603122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360237640,"port_rcv_data":2283824984517,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.327","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.340","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.24,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.354","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.366","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.377","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.389","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.399","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.409","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.2272048,"monotonic":2345794.65929126,"hcas":{"mlx5_0":{"port_xmit_data":98624950273140,"port_rcv_data":100222533644961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827640003,"port_rcv_data":2386041114775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99657221689576,"port_rcv_data":99164243931310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360251683,"port_rcv_data":2283839344197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.307","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.321","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.29,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.334","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.346","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:30.357","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:30.368","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.379","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.388","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.227934,"monotonic":2345795.660020542,"hcas":{"mlx5_0":{"port_xmit_data":98624959492580,"port_rcv_data":100223475549295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827654015,"port_rcv_data":2386055474455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99658147478783,"port_rcv_data":99164267321769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360265602,"port_rcv_data":2283853703877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.327","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.341","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.34,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.355","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":124.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.367","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.378","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.388","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.399","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.408","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.22874,"monotonic":2345796.660826629,"hcas":{"mlx5_0":{"port_xmit_data":98624968920584,"port_rcv_data":100224441052396,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827668027,"port_rcv_data":2386069834135,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99659100918149,"port_rcv_data":99164290976200,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360279598,"port_rcv_data":2283868063557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.308","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.16,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.322","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.79,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.335","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:32.347","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:32.359","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.369","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.380","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.390","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.2288916,"monotonic":2345797.660978117,"hcas":{"mlx5_0":{"port_xmit_data":98624977896267,"port_rcv_data":100225354700449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827682039,"port_rcv_data":2386084193815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99659999774156,"port_rcv_data":99164314136467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360293564,"port_rcv_data":2283882423237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.309","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.323","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.94,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.336","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.348","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.359","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.370","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.380","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.390","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.2296848,"monotonic":2345798.661771221,"hcas":{"mlx5_0":{"port_xmit_data":98624987515063,"port_rcv_data":100226339488613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827696066,"port_rcv_data":2386098553495,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99660968259382,"port_rcv_data":99164337925884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360307576,"port_rcv_data":2283896782917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.312","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.325","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.48,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.338","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.350","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:34.360","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.371","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.381","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.390","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.2303288,"monotonic":2345799.662415369,"hcas":{"mlx5_0":{"port_xmit_data":98624996500080,"port_rcv_data":100227231100413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671827871976,"port_rcv_data":2386107395095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99661851510916,"port_rcv_data":99164373100433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643360833181,"port_rcv_data":2283923307717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.327","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.341","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.84,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.354","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.366","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.377","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.388","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.398","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.408","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.2311373,"monotonic":2345800.663223797,"hcas":{"mlx5_0":{"port_xmit_data":98625006874239,"port_rcv_data":100228351400121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671828443228,"port_rcv_data":2386235257793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99662845573060,"port_rcv_data":99164500671491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361215535,"port_rcv_data":2284041345960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.331","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.345","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.58,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.358","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.370","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.381","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.391","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.401","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.411","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.231947,"monotonic":2345801.664033512,"hcas":{"mlx5_0":{"port_xmit_data":98625016044136,"port_rcv_data":100229266578717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671828598740,"port_rcv_data":2386244237682,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99663750427874,"port_rcv_data":99164537635334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709489,"port_rcv_data":2284069407365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.312","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.326","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.09,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.338","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.349","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.361","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.371","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.381","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.391","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.232265,"monotonic":2345802.664351395,"hcas":{"mlx5_0":{"port_xmit_data":98625025882572,"port_rcv_data":100230243044778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99664701331583,"port_rcv_data":99164546884005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.312","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.326","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.59,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.339","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:38.350","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:38.362","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.372","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.382","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.392","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.232923,"monotonic":2345803.665009848,"hcas":{"mlx5_0":{"port_xmit_data":98625034712579,"port_rcv_data":100231136170865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99665594730356,"port_rcv_data":99164555548587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.332","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.345","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.73,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.358","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.370","gpu_util_pct":98.0,"memory_used_mib":79963.0,"power_w":131.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.381","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.392","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.402","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.14,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.412","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.2335718,"monotonic":2345804.665658279,"hcas":{"mlx5_0":{"port_xmit_data":98625042662886,"port_rcv_data":100232006358457,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99666464650620,"port_rcv_data":99164563338940,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.348","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.361","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.09,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.375","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.387","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.398","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.409","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.419","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.429","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.2343616,"monotonic":2345805.666448197,"hcas":{"mlx5_0":{"port_xmit_data":98625051166578,"port_rcv_data":100232941931852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99667397900016,"port_rcv_data":99164571659139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.332","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.346","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.19,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:41.359","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:41.371","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.382","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.392","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.402","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.411","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.2351687,"monotonic":2345806.667255428,"hcas":{"mlx5_0":{"port_xmit_data":98625059219158,"port_rcv_data":100233823570806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99668281443202,"port_rcv_data":99164579573740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.351","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.364","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.63,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.377","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.389","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.401","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.411","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.421","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.2359378,"monotonic":2345807.668024194,"hcas":{"mlx5_0":{"port_xmit_data":98625067535170,"port_rcv_data":100234736873304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99669194279377,"port_rcv_data":99164587725230,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.316","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.329","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.02,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.342","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:43.353","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:43.364","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.374","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.383","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.392","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.2363546,"monotonic":2345808.668440768,"hcas":{"mlx5_0":{"port_xmit_data":98625075721249,"port_rcv_data":100235634524171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99670090504115,"port_rcv_data":99164595725945,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.352","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.365","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":125.84,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.378","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.390","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.401","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.411","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.422","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.236835,"monotonic":2345809.668921484,"hcas":{"mlx5_0":{"port_xmit_data":98625083640822,"port_rcv_data":100236508470590,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99670966436838,"port_rcv_data":99164603505102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.335","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.349","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.5,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.362","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.374","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.385","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.396","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.406","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.416","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.2372503,"monotonic":2345810.6693367,"hcas":{"mlx5_0":{"port_xmit_data":98625092074465,"port_rcv_data":100237432271883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99671889360489,"port_rcv_data":99164611762844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.352","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.23,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.365","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.58,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.378","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.390","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.401","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.411","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.421","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.238041,"monotonic":2345811.670127285,"hcas":{"mlx5_0":{"port_xmit_data":98625100103140,"port_rcv_data":100238306461940,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99672763506604,"port_rcv_data":99164619621355,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.352","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.366","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.23,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.379","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.390","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":124.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.402","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.412","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.422","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.2388375,"monotonic":2345812.670923838,"hcas":{"mlx5_0":{"port_xmit_data":98625108093292,"port_rcv_data":100239180789363,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99673637169333,"port_rcv_data":99164627444099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.335","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.348","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.94,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.361","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:48.373","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:48.384","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.395","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.405","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.415","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.2396495,"monotonic":2345813.67173597,"hcas":{"mlx5_0":{"port_xmit_data":98625116968173,"port_rcv_data":100240120150050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829059105,"port_rcv_data":2386271176521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99674576913150,"port_rcv_data":99164636170166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361709628,"port_rcv_data":2284069408011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.339","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.89,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.353","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.41,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:49.366","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.378","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.389","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.400","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.409","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.419","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.239937,"monotonic":2345814.672023484,"hcas":{"mlx5_0":{"port_xmit_data":98625126186907,"port_rcv_data":100241068462871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829082588,"port_rcv_data":2386295109321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99675498867471,"port_rcv_data":99164669101412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361733002,"port_rcv_data":2284093340811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.339","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.353","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.96,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.365","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.377","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.388","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.399","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.409","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.419","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.240375,"monotonic":2345815.672461647,"hcas":{"mlx5_0":{"port_xmit_data":98625135548226,"port_rcv_data":100242026329798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829096662,"port_rcv_data":2386309469001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99676444329710,"port_rcv_data":99164692677409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361746999,"port_rcv_data":2284107700491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.356","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.369","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.81,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.382","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.394","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.405","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.416","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.426","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.435","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.24068,"monotonic":2345816.672766346,"hcas":{"mlx5_0":{"port_xmit_data":98625145080601,"port_rcv_data":100243002664191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829110627,"port_rcv_data":2386323828681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99677404017627,"port_rcv_data":99164716370479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361761011,"port_rcv_data":2284122060171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.339","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.353","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.99,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.366","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.72,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.379","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.390","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.401","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.411","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.421","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.2414849,"monotonic":2345817.673571355,"hcas":{"mlx5_0":{"port_xmit_data":98625154552914,"port_rcv_data":100243971035391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829124717,"port_rcv_data":2386338188361,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99678360186001,"port_rcv_data":99164740056586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361775023,"port_rcv_data":2284136419851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.339","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.353","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.73,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.366","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.75,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.378","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.389","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.400","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.410","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.420","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.2422924,"monotonic":2345818.674378945,"hcas":{"mlx5_0":{"port_xmit_data":98625164156088,"port_rcv_data":100244956347981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829142046,"port_rcv_data":2386356093297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99679328865091,"port_rcv_data":99164767427733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361792538,"port_rcv_data":2284154369451,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.356","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.370","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.48,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:54.383","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.395","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.407","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.417","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.428","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.438","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.2426603,"monotonic":2345819.674746456,"hcas":{"mlx5_0":{"port_xmit_data":98625173500997,"port_rcv_data":100245910552446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829152834,"port_rcv_data":2386366907721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99680268325097,"port_rcv_data":99164787347192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643361803202,"port_rcv_data":2284165139211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.356","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.369","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.29,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.382","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.85,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.394","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.405","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.416","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.426","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.436","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.243455,"monotonic":2345820.675541595,"hcas":{"mlx5_0":{"port_xmit_data":98625183044693,"port_rcv_data":100246863471031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829335966,"port_rcv_data":2386378124119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99681212103112,"port_rcv_data":99164825664952,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643362326745,"port_rcv_data":2284194279979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.324","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.337","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.59,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.350","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.04,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.362","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.373","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.383","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.392","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.402","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.2439322,"monotonic":2345821.676018553,"hcas":{"mlx5_0":{"port_xmit_data":98625193414458,"port_rcv_data":100247982073958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671829912504,"port_rcv_data":2386503612019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99682205999129,"port_rcv_data":99164942820739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643362594818,"port_rcv_data":2284301844405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.325","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.338","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.2,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.351","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.363","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.373","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.384","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.393","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.403","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.2447634,"monotonic":2345822.676849928,"hcas":{"mlx5_0":{"port_xmit_data":98625202661583,"port_rcv_data":100248905105406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830067256,"port_rcv_data":2386512591839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99683117995943,"port_rcv_data":99164984576145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363102846,"port_rcv_data":2284334818910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.344","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.357","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.63,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.372","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.02,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.385","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.398","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.410","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.421","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.245399,"monotonic":2345823.677485544,"hcas":{"mlx5_0":{"port_xmit_data":98625212655371,"port_rcv_data":100249894778742,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830527870,"port_rcv_data":2386539530747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99684082800386,"port_rcv_data":99164997042020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363200589,"port_rcv_data":2284337764305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.326","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.339","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.02,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.352","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.36,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.364","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.375","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.385","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.395","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.404","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.2460327,"monotonic":2345824.678119162,"hcas":{"mlx5_0":{"port_xmit_data":98625221612045,"port_rcv_data":100250796621567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830527870,"port_rcv_data":2386539530747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99684984731727,"port_rcv_data":99165005853130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363200589,"port_rcv_data":2284337764305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.360","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.374","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.5,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.386","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.69,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.398","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.409","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.420","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.430","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.439","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.2468255,"monotonic":2345825.678911886,"hcas":{"mlx5_0":{"port_xmit_data":98625230391867,"port_rcv_data":100251721759265,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830527870,"port_rcv_data":2386539530747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99685907257420,"port_rcv_data":99165014426715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363200589,"port_rcv_data":2284337764305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.343","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.356","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.95,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.368","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:01.380","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.391","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.401","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.411","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.420","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.2476392,"monotonic":2345826.679725888,"hcas":{"mlx5_0":{"port_xmit_data":98625238991663,"port_rcv_data":100252614897946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830527870,"port_rcv_data":2386539530747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99686800207646,"port_rcv_data":99165022859524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363201069,"port_rcv_data":2284339297507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.347","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.360","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.89,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.373","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.385","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.395","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.405","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.415","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.425","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.2478628,"monotonic":2345827.679948285,"hcas":{"mlx5_0":{"port_xmit_data":98625248092333,"port_rcv_data":100253552083343,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830551228,"port_rcv_data":2386563463547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99687715683373,"port_rcv_data":99165055733452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363224071,"port_rcv_data":2284361697105,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.364","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.378","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.0,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.391","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:03.402","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.414","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.425","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.435","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.446","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.247932,"monotonic":2345828.6800185,"hcas":{"mlx5_0":{"port_xmit_data":98625257882860,"port_rcv_data":100254556296186,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830565318,"port_rcv_data":2386577823227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99688705158149,"port_rcv_data":99165079709259,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363238176,"port_rcv_data":2284376056785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.384","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.398","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.51,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.410","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.63,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.423","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.434","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.445","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.455","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.465","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.2481682,"monotonic":2345829.680254549,"hcas":{"mlx5_0":{"port_xmit_data":98625266829502,"port_rcv_data":100255468806138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830579268,"port_rcv_data":2386592182907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99689602535134,"port_rcv_data":99165102835727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363252173,"port_rcv_data":2284390416465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.347","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.360","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.76,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.373","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.4,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.385","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.395","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.406","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.415","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.425","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.2489781,"monotonic":2345830.681064688,"hcas":{"mlx5_0":{"port_xmit_data":98625276133465,"port_rcv_data":100256433011558,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830607400,"port_rcv_data":2386620902267,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99690538043754,"port_rcv_data":99165140674388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363280212,"port_rcv_data":2284419135825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.347","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.360","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.1,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.373","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.53,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.385","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.397","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.407","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.418","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.427","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.2497852,"monotonic":2345831.681871612,"hcas":{"mlx5_0":{"port_xmit_data":98625285181340,"port_rcv_data":100257355069189,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830621490,"port_rcv_data":2386635261947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99691445578847,"port_rcv_data":99165163910391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363294193,"port_rcv_data":2284433495505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.330","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.343","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.9,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.356","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.35,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.368","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.379","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.390","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.400","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.410","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.2506123,"monotonic":2345832.682698745,"hcas":{"mlx5_0":{"port_xmit_data":98625294687128,"port_rcv_data":100258316396692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671830621490,"port_rcv_data":2386635261947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99692407458575,"port_rcv_data":99165182101289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363470831,"port_rcv_data":2284442337105,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.347","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.360","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.13,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.373","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.385","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.396","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.406","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.416","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.426","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.2508004,"monotonic":2345833.682886764,"hcas":{"mlx5_0":{"port_xmit_data":98625304741914,"port_rcv_data":100259405808529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831064666,"port_rcv_data":2386752786561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99693377384743,"port_rcv_data":99165308991595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643363920192,"port_rcv_data":2284559864063,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.369","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.382","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.8,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:09.395","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:09.407","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.419","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.429","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.440","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.450","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.2515907,"monotonic":2345834.683676988,"hcas":{"mlx5_0":{"port_xmit_data":98625314478246,"port_rcv_data":100260376158491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831372062,"port_rcv_data":2386770469830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99694332532518,"port_rcv_data":99165335972223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364231153,"port_rcv_data":2284577547275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.332","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.346","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.65,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.359","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.19,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.371","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.381","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.392","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.402","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.411","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.2519362,"monotonic":2345835.684022688,"hcas":{"mlx5_0":{"port_xmit_data":98625324404684,"port_rcv_data":100261366038844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831674947,"port_rcv_data":2386788152898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99695304582074,"port_rcv_data":99165363118511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364532070,"port_rcv_data":2284595230929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.368","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.382","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.94,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.395","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.82,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.406","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.418","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.428","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.438","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.448","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.2527275,"monotonic":2345836.684814202,"hcas":{"mlx5_0":{"port_xmit_data":98625333871002,"port_rcv_data":100262305339446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831975926,"port_rcv_data":2386805835897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99696226313478,"port_rcv_data":99165380967371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364683613,"port_rcv_data":2284604072971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.351","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.364","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.81,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.377","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.06,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.390","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.402","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.413","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.423","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.433","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.25353,"monotonic":2345837.685616665,"hcas":{"mlx5_0":{"port_xmit_data":98625343074464,"port_rcv_data":100263234166077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831975926,"port_rcv_data":2386805835897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99697154140632,"port_rcv_data":99165390009052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364683613,"port_rcv_data":2284604072971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.351","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.365","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.97,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:13.378","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.389","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.401","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.411","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.421","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.254337,"monotonic":2345838.686423538,"hcas":{"mlx5_0":{"port_xmit_data":98625351942957,"port_rcv_data":100264178887048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831975926,"port_rcv_data":2386805835897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99698097293260,"port_rcv_data":99165398701667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364683613,"port_rcv_data":2284604072971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.351","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.364","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.64,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.377","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.389","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.400","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.411","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.421","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.2544236,"monotonic":2345839.686509718,"hcas":{"mlx5_0":{"port_xmit_data":98625360689627,"port_rcv_data":100265090107190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831985366,"port_rcv_data":2386815409017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99699001152790,"port_rcv_data":99165416878269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364693192,"port_rcv_data":2284613646091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.352","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.365","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.26,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.378","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.45,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.390","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.401","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.412","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.422","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.2552357,"monotonic":2345840.68732206,"hcas":{"mlx5_0":{"port_xmit_data":98625370399926,"port_rcv_data":100266084744181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671831999362,"port_rcv_data":2386829768697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99699985018355,"port_rcv_data":99165440777172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364707158,"port_rcv_data":2284628005771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.372","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:16.386","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.0,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:16.399","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.74,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.410","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.421","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.432","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.442","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.452","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.2559333,"monotonic":2345841.688019399,"hcas":{"mlx5_0":{"port_xmit_data":98625379730616,"port_rcv_data":100267038617930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671832013452,"port_rcv_data":2386844128377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99700918229954,"port_rcv_data":99165464267109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364721542,"port_rcv_data":2284642365451,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.336","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.350","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.58,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.363","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.57,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.376","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.387","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.397","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.408","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.418","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.2567577,"monotonic":2345842.688844404,"hcas":{"mlx5_0":{"port_xmit_data":98625389110399,"port_rcv_data":100268010769749,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671832041600,"port_rcv_data":2386872847737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99701861663043,"port_rcv_data":99165502181737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364749674,"port_rcv_data":2284671084811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.337","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.350","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.44,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:18.364","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.375","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.387","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.397","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.407","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.417","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.2575827,"monotonic":2345843.689669286,"hcas":{"mlx5_0":{"port_xmit_data":98625398327211,"port_rcv_data":100268958532713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671832055581,"port_rcv_data":2386887207417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99702796903486,"port_rcv_data":99165525616428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364763702,"port_rcv_data":2284685444491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.355","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.369","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.46,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.382","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.37,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.394","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.405","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.416","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.426","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.435","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.2583919,"monotonic":2345844.690478212,"hcas":{"mlx5_0":{"port_xmit_data":98625407523330,"port_rcv_data":100269892107859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671832069841,"port_rcv_data":2386901567097,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99703716089562,"port_rcv_data":99165549002044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364777714,"port_rcv_data":2284699804171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.355","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:20.368","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.11,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:20.382","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.393","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.404","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.415","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.425","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.434","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.2592018,"monotonic":2345845.691288387,"hcas":{"mlx5_0":{"port_xmit_data":98625417625111,"port_rcv_data":100270832518087,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671832277990,"port_rcv_data":2386910408697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99704647736222,"port_rcv_data":99165567592494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643364954259,"port_rcv_data":2284708645771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.359","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.373","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.68,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.386","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.52,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.398","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.410","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.420","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.430","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.440","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.260009,"monotonic":2345846.692095585,"hcas":{"mlx5_0":{"port_xmit_data":98625428469611,"port_rcv_data":100271904344314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671832721616,"port_rcv_data":2387026438275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99705601948528,"port_rcv_data":99165693777675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643365382539,"port_rcv_data":2284824676245,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.340","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.354","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.03,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.367","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.379","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.390","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.401","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.411","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.421","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.2608333,"monotonic":2345847.692919708,"hcas":{"mlx5_0":{"port_xmit_data":98625438990425,"port_rcv_data":100272847578106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671833037289,"port_rcv_data":2387053556381,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99706519976232,"port_rcv_data":99165730979377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643365697639,"port_rcv_data":2284851794644,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.376","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.390","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.31,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.403","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.415","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.426","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.436","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.447","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.456","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.261624,"monotonic":2345848.693710543,"hcas":{"mlx5_0":{"port_xmit_data":98625450485014,"port_rcv_data":100273787111910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671833379343,"port_rcv_data":2387102476483,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99707410245606,"port_rcv_data":99165788102904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366023449,"port_rcv_data":2284897921362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.359","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.373","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.87,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.386","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.01,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.398","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.410","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.420","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.431","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.441","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.2622662,"monotonic":2345849.694352263,"hcas":{"mlx5_0":{"port_xmit_data":98625461558192,"port_rcv_data":100274678874310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671833523322,"port_rcv_data":2387122745579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99708279986985,"port_rcv_data":99165821929868,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366187129,"port_rcv_data":2284920984721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.359","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.373","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.89,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.385","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.397","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.408","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.418","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.428","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.438","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.24,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.2630773,"monotonic":2345850.695163988,"hcas":{"mlx5_0":{"port_xmit_data":98625472364172,"port_rcv_data":100275579392028,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671833537567,"port_rcv_data":2387137105259,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99709168329956,"port_rcv_data":99165846970337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366201172,"port_rcv_data":2284935344401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.363","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.377","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.07,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.390","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.5,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.401","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.412","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.422","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.432","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.441","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.2638865,"monotonic":2345851.695973016,"hcas":{"mlx5_0":{"port_xmit_data":98625483354011,"port_rcv_data":100276644423383,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671833551579,"port_rcv_data":2387151464939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99710218922463,"port_rcv_data":99165872175673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366215184,"port_rcv_data":2284949704081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.361","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.374","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.33,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.387","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.399","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.410","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.421","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.431","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.441","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.2639318,"monotonic":2345852.696018142,"hcas":{"mlx5_0":{"port_xmit_data":98625494539688,"port_rcv_data":100277739160898,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671833565684,"port_rcv_data":2387165824619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99711298774598,"port_rcv_data":99165897576733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366229165,"port_rcv_data":2284964063761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.380","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.393","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.35,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.407","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.49,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:28.419","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.431","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.442","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.452","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.462","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.2641187,"monotonic":2345853.696205457,"hcas":{"mlx5_0":{"port_xmit_data":98625505897685,"port_rcv_data":100278830728214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671833736897,"port_rcv_data":2387174666219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99712379818506,"port_rcv_data":99165935141508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366731086,"port_rcv_data":2284990588561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.364","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.377","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.45,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.390","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.74,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.402","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.413","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.424","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.434","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.443","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.2649283,"monotonic":2345854.6970147,"hcas":{"mlx5_0":{"port_xmit_data":98625517944703,"port_rcv_data":100279930685766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671834271833,"port_rcv_data":2387264936783,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99713389818779,"port_rcv_data":99166018994402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366950829,"port_rcv_data":2285063175797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.363","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.377","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.21,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.390","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.403","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.414","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.424","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.434","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.444","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.2654054,"monotonic":2345855.697491838,"hcas":{"mlx5_0":{"port_xmit_data":98625529463375,"port_rcv_data":100280918348917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671834302414,"port_rcv_data":2387294048301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99714349829577,"port_rcv_data":99166059440618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643366980512,"port_rcv_data":2285092287379,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.364","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.378","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.16,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.391","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.48,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.402","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.413","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.424","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.434","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.444","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.2662137,"monotonic":2345856.698300306,"hcas":{"mlx5_0":{"port_xmit_data":98625540939938,"port_rcv_data":100281927085819,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671834361965,"port_rcv_data":2387352189557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99715299661895,"port_rcv_data":99166128938526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643367039753,"port_rcv_data":2285150429595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.364","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.37,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.377","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.51,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:32.390","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:32.402","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.414","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.424","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.435","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.445","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.267021,"monotonic":2345857.699107655,"hcas":{"mlx5_0":{"port_xmit_data":98625552517175,"port_rcv_data":100282908169139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671834376427,"port_rcv_data":2387366549237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99716267379909,"port_rcv_data":99166161789859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643367152360,"port_rcv_data":2285171874838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.348","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.362","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.34,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.375","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.3,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.387","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.398","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.409","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.419","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.428","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.267845,"monotonic":2345858.699931135,"hcas":{"mlx5_0":{"port_xmit_data":98625564284450,"port_rcv_data":100283896295129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671834685900,"port_rcv_data":2387398315758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99717223752086,"port_rcv_data":99166206539952,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643367512751,"port_rcv_data":2285205259777,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.368","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.381","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.22,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.394","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.406","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.418","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.429","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.439","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.449","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.2679503,"monotonic":2345859.700036771,"hcas":{"mlx5_0":{"port_xmit_data":98625576151157,"port_rcv_data":100284894137007,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671834996799,"port_rcv_data":2387430082141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99718189834313,"port_rcv_data":99166241031878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643367691125,"port_rcv_data":2285228323351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.368","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.381","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.95,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:35.389","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.5,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:35.398","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.405","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.413","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.421","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.428","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.2681527,"monotonic":2345860.700238293,"hcas":{"mlx5_0":{"port_xmit_data":98625587433884,"port_rcv_data":100285894449988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671834996799,"port_rcv_data":2387430082141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99719190329748,"port_rcv_data":99166269862695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643368058800,"port_rcv_data":2285246006551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.367","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.381","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.94,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.395","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.36,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.408","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.419","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.430","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.441","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.451","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.2689557,"monotonic":2345861.701042203,"hcas":{"mlx5_0":{"port_xmit_data":98625599052941,"port_rcv_data":100287007865976,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671835477593,"port_rcv_data":2387467070811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99720265630325,"port_rcv_data":99166300089151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643368243343,"port_rcv_data":2285265312021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.368","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.381","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.9,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:37.394","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.406","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.418","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.428","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.439","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.449","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.2697616,"monotonic":2345862.701848063,"hcas":{"mlx5_0":{"port_xmit_data":98625610517862,"port_rcv_data":100288128001198,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671835684270,"port_rcv_data":2387529998755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99721324631181,"port_rcv_data":99166374165567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643368453818,"port_rcv_data":2285328239837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.367","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.381","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.93,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.394","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.406","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.417","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.428","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.439","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.449","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.2705662,"monotonic":2345863.702652499,"hcas":{"mlx5_0":{"port_xmit_data":98625623151294,"port_rcv_data":100289051992459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671835714883,"port_rcv_data":2387559542385,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99722216481744,"port_rcv_data":99166416160410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643368484353,"port_rcv_data":2285357783467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.385","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.399","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.99,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.412","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.424","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.436","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.446","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.457","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.467","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.2713544,"monotonic":2345864.703440937,"hcas":{"mlx5_0":{"port_xmit_data":98625636649294,"port_rcv_data":100290051611911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671835761135,"port_rcv_data":2387604262051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99723173622521,"port_rcv_data":99166474257080,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643368530527,"port_rcv_data":2285402504157,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.352","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.366","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.31,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.379","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.391","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.402","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.413","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.423","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.431","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.2721832,"monotonic":2345865.704269829,"hcas":{"mlx5_0":{"port_xmit_data":98625650197358,"port_rcv_data":100291052079037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671835805403,"port_rcv_data":2387648157959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99724129905189,"port_rcv_data":99166531546850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643368574470,"port_rcv_data":2285446400065,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.390","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.404","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.27,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.417","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.430","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.442","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.453","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.463","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.473","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.2725105,"monotonic":2345866.704596702,"hcas":{"mlx5_0":{"port_xmit_data":98625663692477,"port_rcv_data":100292021308041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671835820825,"port_rcv_data":2387662517708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99725083889856,"port_rcv_data":99166568226001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643368791563,"port_rcv_data":2285469739505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.372","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:42.386","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.5,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:42.399","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.62,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.411","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.422","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.433","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.443","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.453","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.2728631,"monotonic":2345867.704949328,"hcas":{"mlx5_0":{"port_xmit_data":98625677458091,"port_rcv_data":100292994873050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671836140513,"port_rcv_data":2387680477072,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99726038448844,"port_rcv_data":99166599505260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643369157456,"port_rcv_data":2285487699455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.409","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.423","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.24,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.436","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.448","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.460","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.470","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.481","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.491","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.2736328,"monotonic":2345868.705719118,"hcas":{"mlx5_0":{"port_xmit_data":98625691267724,"port_rcv_data":100294019325555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671836519224,"port_rcv_data":2387752167861,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99726991624821,"port_rcv_data":99166692523883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643369655843,"port_rcv_data":2285567492271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.355","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.368","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.71,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.381","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.393","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.404","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.415","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.425","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.435","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.274458,"monotonic":2345869.706544152,"hcas":{"mlx5_0":{"port_xmit_data":98625704465760,"port_rcv_data":100295064803467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671836854784,"port_rcv_data":2387794034149,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99727997621132,"port_rcv_data":99166747824362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643370013040,"port_rcv_data":2285609962971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:45.372","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:45.386","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.77,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:45.399","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.412","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.423","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.434","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.444","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.454","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.2752643,"monotonic":2345870.70735065,"hcas":{"mlx5_0":{"port_xmit_data":98625717494588,"port_rcv_data":100296154127413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671837219018,"port_rcv_data":2387864637255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99729015914782,"port_rcv_data":99166813261287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643370066573,"port_rcv_data":2285662884177,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.376","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.390","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.49,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.403","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.416","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.427","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.439","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.449","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.460","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.2760658,"monotonic":2345871.708152175,"hcas":{"mlx5_0":{"port_xmit_data":98625730361262,"port_rcv_data":100297071160795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671837262604,"port_rcv_data":2387907985410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99729889816044,"port_rcv_data":99166877928646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643370257642,"port_rcv_data":2285714865345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.377","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.391","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.02,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.404","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.42,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.416","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.427","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.438","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.448","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.458","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.276869,"monotonic":2345872.708955609,"hcas":{"mlx5_0":{"port_xmit_data":98625743279099,"port_rcv_data":100297976490459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671837438266,"port_rcv_data":2387945339212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99730758128753,"port_rcv_data":99166945184238,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643370729834,"port_rcv_data":2285769488065,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.357","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.371","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.07,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.384","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.62,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.396","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.408","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.419","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.429","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.440","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.2776916,"monotonic":2345873.70977806,"hcas":{"mlx5_0":{"port_xmit_data":98625756596607,"port_rcv_data":100298905907776,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671837908582,"port_rcv_data":2387999961277,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99731630759700,"port_rcv_data":99166986590312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643370757966,"port_rcv_data":2285798208071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.376","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.389","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.0,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.403","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.415","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.426","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.436","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.447","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.456","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.278499,"monotonic":2345874.710585354,"hcas":{"mlx5_0":{"port_xmit_data":98625769166978,"port_rcv_data":100299791439766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671837936575,"port_rcv_data":2388028680637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99732489265852,"port_rcv_data":99167027773880,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643370786068,"port_rcv_data":2285826927431,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.359","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.373","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.81,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.386","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.399","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.410","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.421","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.431","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.441","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.2793212,"monotonic":2345875.711407736,"hcas":{"mlx5_0":{"port_xmit_data":98625781411754,"port_rcv_data":100300732849199,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671838132542,"port_rcv_data":2388051881917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99733407940704,"port_rcv_data":99167071715726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643371141173,"port_rcv_data":2285858970311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.380","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.393","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.55,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.407","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.14,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.419","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.430","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.441","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.452","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.461","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.2801273,"monotonic":2345876.712213895,"hcas":{"mlx5_0":{"port_xmit_data":98625793267273,"port_rcv_data":100301689668532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671838499411,"port_rcv_data":2388094388667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99734322114458,"port_rcv_data":99167125489878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643371512274,"port_rcv_data":2285901477061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:52.380","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:52.393","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.76,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.406","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.418","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.430","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.440","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.455","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.467","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.57,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.280361,"monotonic":2345877.712447041,"hcas":{"mlx5_0":{"port_xmit_data":98625805822646,"port_rcv_data":100302628630544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671838685628,"port_rcv_data":2388133383811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99735220763067,"port_rcv_data":99167194379008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643372049845,"port_rcv_data":2285958155277,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.380","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.394","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.63,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.407","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.75,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.418","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.429","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.439","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.449","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.459","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.2811692,"monotonic":2345878.713255569,"hcas":{"mlx5_0":{"port_xmit_data":98625818978162,"port_rcv_data":100303541531560,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671839179613,"port_rcv_data":2388185498120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99736083824564,"port_rcv_data":99167252511354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643372244416,"port_rcv_data":2286003803791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.362","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.376","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.02,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.389","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.401","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.412","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.422","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.432","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.443","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.281995,"monotonic":2345879.714081677,"hcas":{"mlx5_0":{"port_xmit_data":98625832008715,"port_rcv_data":100304505859124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671839425567,"port_rcv_data":2388278561175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99736952887657,"port_rcv_data":99167338149370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643372320459,"port_rcv_data":2286076810037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.380","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.394","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.92,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.406","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.42,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.418","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.429","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.439","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.449","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.459","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.2828035,"monotonic":2345880.714890073,"hcas":{"mlx5_0":{"port_xmit_data":98625844726119,"port_rcv_data":100305412748452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671839472873,"port_rcv_data":2388323194341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99737817050254,"port_rcv_data":99167395383034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643372366711,"port_rcv_data":2286121385461,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:56.379","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.393","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.92,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.406","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.419","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.430","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.441","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.452","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.462","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.28361,"monotonic":2345881.71569668,"hcas":{"mlx5_0":{"port_xmit_data":98625857704363,"port_rcv_data":100306337615683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671839674234,"port_rcv_data":2388376669581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99738688563631,"port_rcv_data":99167469547011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643372697574,"port_rcv_data":2286182919994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.364","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.377","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.47,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.390","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.402","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.412","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.423","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.433","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.443","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.2839353,"monotonic":2345882.7160217,"hcas":{"mlx5_0":{"port_xmit_data":98625871000033,"port_rcv_data":100307210930503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671840010770,"port_rcv_data":2388424027890,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99739514525991,"port_rcv_data":99167530732644,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643373060600,"port_rcv_data":2286231257834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:58.384","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.398","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.61,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.411","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.36,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.424","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.435","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.446","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.456","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.465","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.2847412,"monotonic":2345883.716827684,"hcas":{"mlx5_0":{"port_xmit_data":98625884858637,"port_rcv_data":100307983033158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671840193096,"port_rcv_data":2388461726932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99740249098281,"port_rcv_data":99167581939206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643373252769,"port_rcv_data":2286268896403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.384","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.398","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.65,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.411","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.422","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.433","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.443","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.453","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.463","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.2855499,"monotonic":2345884.717636265,"hcas":{"mlx5_0":{"port_xmit_data":98625898125089,"port_rcv_data":100308857471500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671840549798,"port_rcv_data":2388479564764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99741104120403,"port_rcv_data":99167612512325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643373602759,"port_rcv_data":2286286734821,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.401","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.414","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.04,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.427","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.439","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.450","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.460","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.470","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.479","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.2863424,"monotonic":2345885.718428779,"hcas":{"mlx5_0":{"port_xmit_data":98625911296548,"port_rcv_data":100309771957110,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671840927858,"port_rcv_data":2388520538997,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99741980249214,"port_rcv_data":99167662950731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643373955694,"port_rcv_data":2286324532835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:01.384","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.397","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.71,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.410","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.422","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.434","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.444","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.454","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.464","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.2871504,"monotonic":2345886.719236832,"hcas":{"mlx5_0":{"port_xmit_data":98625923915930,"port_rcv_data":100310685693422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671841238478,"port_rcv_data":2388564529433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99742848400313,"port_rcv_data":99167731112345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643374458328,"port_rcv_data":2286380465523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.387","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.400","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.46,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.414","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.426","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.438","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.449","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.459","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.469","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.2879398,"monotonic":2345887.720026265,"hcas":{"mlx5_0":{"port_xmit_data":98625935667207,"port_rcv_data":100311644340943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671841580176,"port_rcv_data":2388611678821,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99743761483016,"port_rcv_data":99167771833852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643374488491,"port_rcv_data":2286409930411,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.388","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.401","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.84,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.414","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.95,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.427","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.439","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.450","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.460","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.470","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.2887447,"monotonic":2345888.720830962,"hcas":{"mlx5_0":{"port_xmit_data":98625947077956,"port_rcv_data":100312608426918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671841640982,"port_rcv_data":2388670766081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99744664350829,"port_rcv_data":99167842092508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643374549855,"port_rcv_data":2286469017735,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.388","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.401","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.31,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.414","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.82,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.426","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.437","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.448","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.458","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.468","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.2895508,"monotonic":2345889.721637329,"hcas":{"mlx5_0":{"port_xmit_data":98625958765001,"port_rcv_data":100313570204195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671841671393,"port_rcv_data":2388700301797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99745597601002,"port_rcv_data":99167883193922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643374580189,"port_rcv_data":2286498554411,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.387","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.401","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.95,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.414","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.15,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.426","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.438","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.448","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.459","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.469","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.2903569,"monotonic":2345890.722443466,"hcas":{"mlx5_0":{"port_xmit_data":98625970242086,"port_rcv_data":100314488615844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671841702099,"port_rcv_data":2388729838025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99746487067327,"port_rcv_data":99167924073533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643374610724,"port_rcv_data":2286528090639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:06.370","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.384","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.53,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.398","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":139.76,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.410","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.422","gpu_util_pct":99.0,"memory_used_mib":79915.0,"power_w":133.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.433","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.444","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.454","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.2911808,"monotonic":2345891.723267525,"hcas":{"mlx5_0":{"port_xmit_data":98625983118156,"port_rcv_data":100315377302693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671841776158,"port_rcv_data":2388745636305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99747360625015,"port_rcv_data":99167955335189,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643374824035,"port_rcv_data":2286546643519,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.408","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.422","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.84,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.435","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.34,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.448","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.459","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.470","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.480","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.490","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.291935,"monotonic":2345892.724021362,"hcas":{"mlx5_0":{"port_xmit_data":98625996407875,"port_rcv_data":100316299299726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671842083988,"port_rcv_data":2388782135370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99748244535071,"port_rcv_data":99168019773586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643375330156,"port_rcv_data":2286598349192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.409","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.423","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.69,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.436","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.79,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.449","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.460","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.471","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.481","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.491","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.2927237,"monotonic":2345893.724810049,"hcas":{"mlx5_0":{"port_xmit_data":98626009689300,"port_rcv_data":100317227202258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671842572300,"port_rcv_data":2388838260503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99749115979775,"port_rcv_data":99168070617337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643375515366,"port_rcv_data":2286636515461,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.391","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.405","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.84,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.417","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.66,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.429","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.439","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.450","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.460","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.469","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.2935326,"monotonic":2345894.725619308,"hcas":{"mlx5_0":{"port_xmit_data":98626022161132,"port_rcv_data":100318172080939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671842759974,"port_rcv_data":2388876438803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99750024811220,"port_rcv_data":99168138678619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643376007708,"port_rcv_data":2286692352037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.392","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.405","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.74,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.418","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.5,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.430","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.442","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.453","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.463","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.472","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.2943408,"monotonic":2345895.726427301,"hcas":{"mlx5_0":{"port_xmit_data":98626034303818,"port_rcv_data":100319191417680,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671843147148,"port_rcv_data":2388952948625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99750967914078,"port_rcv_data":99168209161516,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643376068917,"port_rcv_data":2286751236435,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.375","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.388","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.4,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.402","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.19,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.414","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.426","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.436","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.19,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.447","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.456","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.2947452,"monotonic":2345896.726831331,"hcas":{"mlx5_0":{"port_xmit_data":98626046169510,"port_rcv_data":100320168931446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671843177513,"port_rcv_data":2388982376116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99751913710102,"port_rcv_data":99168250263209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643376099204,"port_rcv_data":2286780664406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.396","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.409","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.82,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.422","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.01,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.434","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.445","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.455","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.466","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.475","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.2955492,"monotonic":2345897.727635805,"hcas":{"mlx5_0":{"port_xmit_data":98626057537753,"port_rcv_data":100321088986435,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671843193726,"port_rcv_data":2388997444183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99752820955973,"port_rcv_data":99168285426664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643376291947,"port_rcv_data":2286804574073,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:13.395","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.409","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.46,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.422","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.434","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.446","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.456","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.467","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.477","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.295934,"monotonic":2345898.7280205,"hcas":{"mlx5_0":{"port_xmit_data":98626069237553,"port_rcv_data":100322025396084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671843516916,"port_rcv_data":2389025532495,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99753729503739,"port_rcv_data":99168324676158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643376665543,"port_rcv_data":2286832663557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.393","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.407","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.73,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.419","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.34,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.432","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.443","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.453","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.463","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.473","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.2967432,"monotonic":2345899.728829762,"hcas":{"mlx5_0":{"port_xmit_data":98626081517632,"port_rcv_data":100323032440815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671844024310,"port_rcv_data":2389091636778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99754670512873,"port_rcv_data":99168384658119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643376859898,"port_rcv_data":2286881007608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.378","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.392","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.56,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.405","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.78,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.417","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.429","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.439","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.450","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.460","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.2974036,"monotonic":2345900.729490084,"hcas":{"mlx5_0":{"port_xmit_data":98626093357192,"port_rcv_data":100324042621131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671844092479,"port_rcv_data":2389159357281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99755611367167,"port_rcv_data":99168499594602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643377549772,"port_rcv_data":2286984403809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:16.413","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.426","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.87,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.440","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.06,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.451","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.462","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.473","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.483","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.492","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.2977357,"monotonic":2345901.729822111,"hcas":{"mlx5_0":{"port_xmit_data":98626105731229,"port_rcv_data":100325062371993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671844597794,"port_rcv_data":2389229901249,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99756560522215,"port_rcv_data":99168555112643,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643377594164,"port_rcv_data":2287028191915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.395","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.409","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.55,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.422","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.434","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.445","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.456","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.466","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.476","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.2985423,"monotonic":2345902.730628949,"hcas":{"mlx5_0":{"port_xmit_data":98626117481897,"port_rcv_data":100326043253142,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671844642202,"port_rcv_data":2389273688420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99757498319542,"port_rcv_data":99168610496645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643377638277,"port_rcv_data":2287071979566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.378","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.392","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.9,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.405","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.13,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.418","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.429","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.440","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.451","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.460","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.2993639,"monotonic":2345903.731450318,"hcas":{"mlx5_0":{"port_xmit_data":98626128126682,"port_rcv_data":100327048501547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671844686299,"port_rcv_data":2389317475847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99758458862144,"port_rcv_data":99168664742087,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643377682622,"port_rcv_data":2287115766993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.400","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.414","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.64,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.427","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.2,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.439","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.450","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.461","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.471","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.481","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.3001678,"monotonic":2345904.732254015,"hcas":{"mlx5_0":{"port_xmit_data":98626139156828,"port_rcv_data":100328084732503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671844876670,"port_rcv_data":2389354907174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99759460317132,"port_rcv_data":99168730897948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643378231229,"port_rcv_data":2287171243333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.417","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.430","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.51,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.443","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.37,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.455","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.466","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.477","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.487","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.497","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.300956,"monotonic":2345905.733042425,"hcas":{"mlx5_0":{"port_xmit_data":98626150825667,"port_rcv_data":100329082516579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671845420658,"port_rcv_data":2389396230551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99760414619024,"port_rcv_data":99168765129755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643378415478,"port_rcv_data":2287194522869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:21.381","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.395","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.18,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.408","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.46,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.421","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.432","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.443","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.453","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.463","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.30178,"monotonic":2345906.733866626,"hcas":{"mlx5_0":{"port_xmit_data":98626162522810,"port_rcv_data":100330038341953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671845605930,"port_rcv_data":2389432547030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99761336653283,"port_rcv_data":99168830407809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643378921243,"port_rcv_data":2287248689086,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.382","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.396","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.76,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.408","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.02,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.420","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.431","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.442","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.452","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.464","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.3026018,"monotonic":2345907.734688356,"hcas":{"mlx5_0":{"port_xmit_data":98626173943561,"port_rcv_data":100331081907722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671846116236,"port_rcv_data":2389483438834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99762329611366,"port_rcv_data":99168871369277,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643379099508,"port_rcv_data":2287278800502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.399","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.412","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.61,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.426","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.21,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.438","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.449","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.460","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.469","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.479","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.3034089,"monotonic":2345908.735495459,"hcas":{"mlx5_0":{"port_xmit_data":98626184879667,"port_rcv_data":100332088507261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671846410907,"port_rcv_data":2389512466714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99763305612730,"port_rcv_data":99168922727006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643379572630,"port_rcv_data":2287319684982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.403","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.417","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.12,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.430","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.01,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.441","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.452","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.463","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.473","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.483","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.3039336,"monotonic":2345909.736020339,"hcas":{"mlx5_0":{"port_xmit_data":98626195884525,"port_rcv_data":100333113675486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671846806591,"port_rcv_data":2389562132419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99764280795474,"port_rcv_data":99168977216733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643379857799,"port_rcv_data":2287363772465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.420","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.434","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.11,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.447","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.43,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.460","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.471","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.482","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.492","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.502","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.3047214,"monotonic":2345910.736807915,"hcas":{"mlx5_0":{"port_xmit_data":98626207120036,"port_rcv_data":100334158638369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671846982345,"port_rcv_data":2389586281258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99765303932150,"port_rcv_data":99169008998294,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643379978219,"port_rcv_data":2287384659098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.385","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.398","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.1,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.412","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.424","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.436","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.447","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.73,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.457","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.467","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.3055422,"monotonic":2345911.737628683,"hcas":{"mlx5_0":{"port_xmit_data":98626218090645,"port_rcv_data":100335182409375,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671846996373,"port_rcv_data":2389600640938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99766312537953,"port_rcv_data":99169034176124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643379992246,"port_rcv_data":2287399018778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.403","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.416","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.34,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.429","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.77,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.441","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.452","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.463","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.473","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.483","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.3063514,"monotonic":2345912.738437843,"hcas":{"mlx5_0":{"port_xmit_data":98626228633498,"port_rcv_data":100336205827891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671847010493,"port_rcv_data":2389615000618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99767319863173,"port_rcv_data":99169058906599,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643380006258,"port_rcv_data":2287413378458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.404","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.417","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.37,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.431","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.52,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.443","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.455","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.465","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.476","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.486","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.3071566,"monotonic":2345913.739243004,"hcas":{"mlx5_0":{"port_xmit_data":98626238372663,"port_rcv_data":100337180493604,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671847208816,"port_rcv_data":2389623842218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99768286372912,"port_rcv_data":99169085989172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643380352528,"port_rcv_data":2287431061658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.403","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.417","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.38,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.430","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.442","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.453","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.464","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.474","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.483","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.3079677,"monotonic":2345914.740054449,"hcas":{"mlx5_0":{"port_xmit_data":98626248093473,"port_rcv_data":100338237246339,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671847619194,"port_rcv_data":2389738367174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99769230567978,"port_rcv_data":99169209653425,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643380767324,"port_rcv_data":2287545588298,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.389","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.402","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.81,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.416","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.35,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.428","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.440","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.450","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.461","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.471","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.3087914,"monotonic":2345915.74087783,"hcas":{"mlx5_0":{"port_xmit_data":98626258406521,"port_rcv_data":100339274805629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671847921273,"port_rcv_data":2389755769988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99770248843799,"port_rcv_data":99169245207889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381211879,"port_rcv_data":2287571271974,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.388","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.402","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.68,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.415","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.89,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.427","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.438","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.448","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.458","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.468","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.3096197,"monotonic":2345916.741706242,"hcas":{"mlx5_0":{"port_xmit_data":98626268327100,"port_rcv_data":100340260560971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848371254,"port_rcv_data":2389781453078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99771211011286,"port_rcv_data":99169263143468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381357099,"port_rcv_data":2287579833774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.407","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.420","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.55,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.433","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.14,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.445","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.456","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.467","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.477","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.486","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.3104348,"monotonic":2345917.742521297,"hcas":{"mlx5_0":{"port_xmit_data":98626277854217,"port_rcv_data":100341231658338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848371254,"port_rcv_data":2389781453078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99772181427191,"port_rcv_data":99169272519320,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381357099,"port_rcv_data":2287579833774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.393","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":143.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.407","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.49,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.420","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.53,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.433","gpu_util_pct":99.0,"memory_used_mib":79963.0,"power_w":137.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.445","gpu_util_pct":98.0,"memory_used_mib":79915.0,"power_w":134.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.456","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.465","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.475","gpu_util_pct":98.0,"memory_used_mib":80045.0,"power_w":129.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.3112578,"monotonic":2345918.743344259,"hcas":{"mlx5_0":{"port_xmit_data":98626285252664,"port_rcv_data":100342039187514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848371254,"port_rcv_data":2389781453078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99772987192766,"port_rcv_data":99169279766125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381357099,"port_rcv_data":2287579833774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.411","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.425","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":125.95,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.438","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.451","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.462","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":124.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.473","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.22,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.483","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.492","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":122.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.3120644,"monotonic":2345919.744151272,"hcas":{"mlx5_0":{"port_xmit_data":98626293308993,"port_rcv_data":100342924036636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848371254,"port_rcv_data":2389781453078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99773874161649,"port_rcv_data":99169287705241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381357099,"port_rcv_data":2287579833774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.412","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.426","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.23,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.439","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.03,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.450","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.461","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":124.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.471","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.481","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.491","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.312874,"monotonic":2345920.744960592,"hcas":{"mlx5_0":{"port_xmit_data":98626302115463,"port_rcv_data":100343858641211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848371254,"port_rcv_data":2389781453078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99774808478729,"port_rcv_data":99169296372166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381357099,"port_rcv_data":2287579833774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.413","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.427","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.71,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.440","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.07,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.453","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.464","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.475","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.485","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.495","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.31368,"monotonic":2345921.745766576,"hcas":{"mlx5_0":{"port_xmit_data":98626311919802,"port_rcv_data":100344876531430,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848394597,"port_rcv_data":2389805385878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99775801890733,"port_rcv_data":99169329942304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381380519,"port_rcv_data":2287603766574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.394","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.408","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.73,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.421","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.82,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.434","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.445","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.456","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.466","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.476","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.3145063,"monotonic":2345922.746592527,"hcas":{"mlx5_0":{"port_xmit_data":98626321970797,"port_rcv_data":100345912482443,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848408671,"port_rcv_data":2389819745558,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99776821587326,"port_rcv_data":99169354168199,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381394857,"port_rcv_data":2287618126254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.411","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.425","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.14,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.438","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.449","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.460","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.471","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.481","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.490","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.3153203,"monotonic":2345923.747406708,"hcas":{"mlx5_0":{"port_xmit_data":98626331510675,"port_rcv_data":100346892345414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848422652,"port_rcv_data":2389834105238,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99777789207700,"port_rcv_data":99169377944236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381408869,"port_rcv_data":2287632485934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.416","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.430","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.32,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.442","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.3,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.454","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.464","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.475","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.486","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.496","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.3159354,"monotonic":2345924.748021873,"hcas":{"mlx5_0":{"port_xmit_data":98626340500605,"port_rcv_data":100347812118708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848437051,"port_rcv_data":2389848464918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99778692210498,"port_rcv_data":99169401104568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381422881,"port_rcv_data":2287646845614,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.432","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.446","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.3,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.459","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.81,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.471","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.482","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.493","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.503","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.512","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.316729,"monotonic":2345925.748815603,"hcas":{"mlx5_0":{"port_xmit_data":98626351031965,"port_rcv_data":100348943175750,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848465029,"port_rcv_data":2389877184278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99779796812274,"port_rcv_data":99169440229895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381450951,"port_rcv_data":2287675564974,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.416","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.429","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.49,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.442","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.455","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.466","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.477","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.487","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.496","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.3170536,"monotonic":2345926.749139766,"hcas":{"mlx5_0":{"port_xmit_data":98626362242942,"port_rcv_data":100350139872964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848653013,"port_rcv_data":2389886025878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99780984541194,"port_rcv_data":99169451160907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643381450951,"port_rcv_data":2287675564974,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.416","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.430","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.78,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.443","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.49,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.455","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.467","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.477","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.488","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.498","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.3178625,"monotonic":2345927.749948765,"hcas":{"mlx5_0":{"port_xmit_data":98626373635982,"port_rcv_data":100351445892637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671848919954,"port_rcv_data":2389993994574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99782181243631,"port_rcv_data":99169594206452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382128394,"port_rcv_data":2287807924394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.416","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.429","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.13,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.443","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.455","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.466","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.476","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.487","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.496","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.3186707,"monotonic":2345928.750757462,"hcas":{"mlx5_0":{"port_xmit_data":98626385249291,"port_rcv_data":100352672108542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849397307,"port_rcv_data":2390020466850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99783380782516,"port_rcv_data":99169616312601,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382359034,"port_rcv_data":2287818849523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.416","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.13,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.430","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.45,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.443","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.43,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.456","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.468","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.478","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.3,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.489","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.499","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.319477,"monotonic":2345929.751563498,"hcas":{"mlx5_0":{"port_xmit_data":98626396572548,"port_rcv_data":100353883613455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849575015,"port_rcv_data":2390032289394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99784580897602,"port_rcv_data":99169653622268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805357,"port_rcv_data":2287845217199,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.410","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.424","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.85,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.437","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.68,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.449","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.461","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.471","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.481","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.491","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.3199353,"monotonic":2345930.752021714,"hcas":{"mlx5_0":{"port_xmit_data":98626407985433,"port_rcv_data":100355097921044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99785780538326,"port_rcv_data":99169664656696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.402","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.416","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.54,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.429","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.44,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.441","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.452","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.462","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.472","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.482","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.320759,"monotonic":2345931.752845419,"hcas":{"mlx5_0":{"port_xmit_data":98626418767753,"port_rcv_data":100356310988345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99786993474188,"port_rcv_data":99169675330392,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.401","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.415","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.33,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.428","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.57,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.441","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.453","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.464","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.474","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.484","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.3210824,"monotonic":2345932.753168842,"hcas":{"mlx5_0":{"port_xmit_data":98626429421770,"port_rcv_data":100357571267381,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99788253626337,"port_rcv_data":99169685882245,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.419","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.433","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.1,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.446","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.458","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.470","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.480","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.491","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.500","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.3218925,"monotonic":2345933.753979077,"hcas":{"mlx5_0":{"port_xmit_data":98626439893605,"port_rcv_data":100358807603090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99789492551911,"port_rcv_data":99169696272331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.419","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.432","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.72,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.445","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.43,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.457","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.468","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.479","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.489","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.499","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.3227072,"monotonic":2345934.75479363,"hcas":{"mlx5_0":{"port_xmit_data":98626450433546,"port_rcv_data":100360055041042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99790739805125,"port_rcv_data":99169706718126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.420","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.433","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.29,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.446","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.41,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.458","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.469","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.479","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.489","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.498","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.3235178,"monotonic":2345935.755604181,"hcas":{"mlx5_0":{"port_xmit_data":98626461030851,"port_rcv_data":100361308975221,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99791993383059,"port_rcv_data":99169717208575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.440","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.454","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.65,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.467","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.45,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.478","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.489","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.500","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.509","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.519","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.32431,"monotonic":2345936.756396528,"hcas":{"mlx5_0":{"port_xmit_data":98626471716559,"port_rcv_data":100362573182243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99793257365318,"port_rcv_data":99169727793761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.424","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.06,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.437","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.27,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.450","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.44,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.464","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.476","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:52.488","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:52.498","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:52.509","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.3251135,"monotonic":2345937.75720023,"hcas":{"mlx5_0":{"port_xmit_data":98626482274711,"port_rcv_data":100363820087774,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99794504543196,"port_rcv_data":99169738251872,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.424","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.437","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.63,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.450","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.76,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.462","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.473","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.483","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.493","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.502","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.3259263,"monotonic":2345938.758012857,"hcas":{"mlx5_0":{"port_xmit_data":98626492904443,"port_rcv_data":100365078002316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99795762102642,"port_rcv_data":99169748783440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.423","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.437","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.48,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.450","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.463","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.474","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.485","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.496","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:54.506","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.3267362,"monotonic":2345939.758822447,"hcas":{"mlx5_0":{"port_xmit_data":98626503498797,"port_rcv_data":100366331442960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99797014102425,"port_rcv_data":99169759266726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.424","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":165.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.438","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.26,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.451","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.463","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.475","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.485","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.495","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.506","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.3275433,"monotonic":2345940.759629713,"hcas":{"mlx5_0":{"port_xmit_data":98626514139918,"port_rcv_data":100367590247716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99798274252297,"port_rcv_data":99169769816921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.408","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.421","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.49,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.434","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.48,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.446","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.457","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.468","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.478","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.488","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.3283696,"monotonic":2345941.760456248,"hcas":{"mlx5_0":{"port_xmit_data":98626524571724,"port_rcv_data":100368820188829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99799504133763,"port_rcv_data":99169780153666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.428","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.442","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.51,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.455","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.86,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.467","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.479","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.490","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.500","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.509","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.329176,"monotonic":2345942.761262436,"hcas":{"mlx5_0":{"port_xmit_data":98626535167616,"port_rcv_data":100370073887334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99800755217316,"port_rcv_data":99169790625815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.445","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.458","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.63,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.472","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.41,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.484","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.495","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.506","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.73,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.516","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.526","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.329965,"monotonic":2345943.762051836,"hcas":{"mlx5_0":{"port_xmit_data":98626545814370,"port_rcv_data":100371333507453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99802015370089,"port_rcv_data":99169801179376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.427","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.441","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.28,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.454","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.38,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.466","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.477","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.488","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.498","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.508","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.330769,"monotonic":2345944.762855649,"hcas":{"mlx5_0":{"port_xmit_data":98626556311578,"port_rcv_data":100372580726769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99803264617310,"port_rcv_data":99169811604737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.431","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":164.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.445","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.68,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.459","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.471","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.482","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.493","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.503","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.513","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.331577,"monotonic":2345945.763663477,"hcas":{"mlx5_0":{"port_xmit_data":98626567066036,"port_rcv_data":100373844816537,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99804528369442,"port_rcv_data":99169822245381,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.431","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":165.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.445","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.13,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.458","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.13,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.470","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.481","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.492","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.502","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.512","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.3319323,"monotonic":2345946.764018842,"hcas":{"mlx5_0":{"port_xmit_data":98626577709689,"port_rcv_data":100375103959593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99805786786311,"port_rcv_data":99169832790107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.431","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.445","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.36,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.458","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.470","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.481","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.492","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.502","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.511","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.332741,"monotonic":2345947.764827536,"hcas":{"mlx5_0":{"port_xmit_data":98626588302144,"port_rcv_data":100376354320461,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99807034783998,"port_rcv_data":99169843258752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.448","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.461","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.96,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.474","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.89,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.486","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.498","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.509","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.519","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.528","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.333536,"monotonic":2345948.765622327,"hcas":{"mlx5_0":{"port_xmit_data":98626598969568,"port_rcv_data":100377571630623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849850372,"port_rcv_data":2390046834078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99808253554194,"port_rcv_data":99169853842468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382805450,"port_rcv_data":2287845217630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.414","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.427","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.65,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.440","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.452","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.463","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.473","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.483","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.493","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.3343658,"monotonic":2345949.766452358,"hcas":{"mlx5_0":{"port_xmit_data":98626609944053,"port_rcv_data":100378785813924,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849883232,"port_rcv_data":2390080339998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99809435186530,"port_rcv_data":99169898199420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382838387,"port_rcv_data":2287878723550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.432","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.445","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.18,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.458","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.74,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.470","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.482","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.493","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.42,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.503","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.512","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.3351755,"monotonic":2345950.76726198,"hcas":{"mlx5_0":{"port_xmit_data":98626620865845,"port_rcv_data":100379983026681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849911318,"port_rcv_data":2390109059358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99810600934784,"port_rcv_data":99169937678543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382866597,"port_rcv_data":2287907442910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.435","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.445","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.82,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.454","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.12,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.463","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.472","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.480","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.488","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.497","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.3360074,"monotonic":2345951.768093778,"hcas":{"mlx5_0":{"port_xmit_data":98626631772904,"port_rcv_data":100381183389456,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849939311,"port_rcv_data":2390137778718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99811774404740,"port_rcv_data":99169977200905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382894606,"port_rcv_data":2287936162270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.435","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.449","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.71,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.462","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.475","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.486","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.497","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:07.517","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.3368118,"monotonic":2345952.768898252,"hcas":{"mlx5_0":{"port_xmit_data":98626642753582,"port_rcv_data":100382390053666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849967397,"port_rcv_data":2390166498078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99812952632026,"port_rcv_data":99170016761458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382922599,"port_rcv_data":2287964881630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.435","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.449","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.6,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.462","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.474","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.486","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.496","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.506","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.516","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.3376203,"monotonic":2345953.769706781,"hcas":{"mlx5_0":{"port_xmit_data":98626653665777,"port_rcv_data":100383591328697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671849995406,"port_rcv_data":2390195217438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99814125441405,"port_rcv_data":99170056274458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382950576,"port_rcv_data":2287993600990,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.419","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.432","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.97,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.445","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.51,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.457","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.468","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.479","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.489","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.498","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.3384473,"monotonic":2345954.77053387,"hcas":{"mlx5_0":{"port_xmit_data":98626664637110,"port_rcv_data":100384797740061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671850023709,"port_rcv_data":2390223936798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99815302648058,"port_rcv_data":99170095827213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643382978662,"port_rcv_data":2288022320350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.452","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.466","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.96,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.480","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.57,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.492","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.503","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.514","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.525","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.534","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.3392377,"monotonic":2345955.771324191,"hcas":{"mlx5_0":{"port_xmit_data":98626675726809,"port_rcv_data":100385995380445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671850207043,"port_rcv_data":2390247138078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99816477824567,"port_rcv_data":99170141485271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643383393783,"port_rcv_data":2288057209270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.440","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.453","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.38,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.467","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.92,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.479","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.490","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.501","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.512","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.522","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.3400424,"monotonic":2345956.772128503,"hcas":{"mlx5_0":{"port_xmit_data":98626687166169,"port_rcv_data":100387223085586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671850608136,"port_rcv_data":2390313262786,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99817641742224,"port_rcv_data":99170213205529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643383740069,"port_rcv_data":2288118065982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.420","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.434","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.46,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.447","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.83,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.460","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.471","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.482","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.492","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.502","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.340867,"monotonic":2345957.77295333,"hcas":{"mlx5_0":{"port_xmit_data":98626698932189,"port_rcv_data":100388363633957,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671850924321,"port_rcv_data":2390338095466,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99818755576983,"port_rcv_data":99170255194299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643384096925,"port_rcv_data":2288148759702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.421","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.430","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.81,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.439","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.1,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.448","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.457","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.466","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.474","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.482","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.3417149,"monotonic":2345958.7738015,"hcas":{"mlx5_0":{"port_xmit_data":98626711181548,"port_rcv_data":100389440893015,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671851409316,"port_rcv_data":2390378979946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99819791972496,"port_rcv_data":99170295539741,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643384377429,"port_rcv_data":2288177365182,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.439","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.453","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.31,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.466","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.05,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.479","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.490","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.501","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.511","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.520","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.3425226,"monotonic":2345959.774609319,"hcas":{"mlx5_0":{"port_xmit_data":98626723514147,"port_rcv_data":100390458232871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671851693307,"port_rcv_data":2390423221690,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99820765152857,"port_rcv_data":99170351640441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643384663311,"port_rcv_data":2288221608098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.457","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.471","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.96,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.485","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.497","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.508","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.519","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.529","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.539","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.3433127,"monotonic":2345960.775399199,"hcas":{"mlx5_0":{"port_xmit_data":98626736172756,"port_rcv_data":100391373019310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671851707505,"port_rcv_data":2390437581370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99821665394076,"port_rcv_data":99170378571341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643384677540,"port_rcv_data":2288235967778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.443","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.457","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.68,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.471","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.483","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.495","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.507","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.94,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.517","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.527","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.3441145,"monotonic":2345961.776200942,"hcas":{"mlx5_0":{"port_xmit_data":98626748664821,"port_rcv_data":100392273274964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671851721517,"port_rcv_data":2390451941050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99822549004604,"port_rcv_data":99170405298873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643384691552,"port_rcv_data":2288250327458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.444","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.456","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.0,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.464","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.06,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.473","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.481","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.489","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.497","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.505","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.344943,"monotonic":2345962.777029619,"hcas":{"mlx5_0":{"port_xmit_data":98626761289710,"port_rcv_data":100393185781803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671851735761,"port_rcv_data":2390466300730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99823447059361,"port_rcv_data":99170432199475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643384705688,"port_rcv_data":2288264687138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.461","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.474","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.4,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.487","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.499","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.510","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.521","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.38,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.532","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.542","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.3457313,"monotonic":2345963.77781777,"hcas":{"mlx5_0":{"port_xmit_data":98626773945909,"port_rcv_data":100394083668682,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671851908881,"port_rcv_data":2390475142330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99824336363159,"port_rcv_data":99170462299795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643385071209,"port_rcv_data":2288282370338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.443","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.457","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.19,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.471","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.483","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.495","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.505","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.516","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.526","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.3465455,"monotonic":2345964.778631783,"hcas":{"mlx5_0":{"port_xmit_data":98626786956577,"port_rcv_data":100395078913618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671852307975,"port_rcv_data":2390572352312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99825236024099,"port_rcv_data":99170571960154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643385468629,"port_rcv_data":2288379582664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.461","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.475","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.88,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.488","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.501","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.512","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.523","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.534","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.543","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.3473425,"monotonic":2345965.779428716,"hcas":{"mlx5_0":{"port_xmit_data":98626800159138,"port_rcv_data":100395933315410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671852504856,"port_rcv_data":2390620069938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99826043581100,"port_rcv_data":99170623826847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643385508510,"port_rcv_data":2288418459990,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.448","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.461","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.42,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.474","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.487","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.498","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.508","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.518","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.36,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:21.528","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.3481512,"monotonic":2345966.780237619,"hcas":{"mlx5_0":{"port_xmit_data":98626813580599,"port_rcv_data":100396726029928,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671852548969,"port_rcv_data":2390663733824,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99826790362597,"port_rcv_data":99170680785403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643385552592,"port_rcv_data":2288462122576,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.469","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.483","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.95,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.496","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.508","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.519","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.540","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.550","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.3489356,"monotonic":2345967.781022117,"hcas":{"mlx5_0":{"port_xmit_data":98626826837550,"port_rcv_data":100397497570373,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671852586572,"port_rcv_data":2390702026304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99827524681684,"port_rcv_data":99170732268571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643385590149,"port_rcv_data":2288500415056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.430","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.444","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.83,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.457","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.470","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.481","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:23.492","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:23.502","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:23.511","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.349757,"monotonic":2345968.781843509,"hcas":{"mlx5_0":{"port_xmit_data":98626839433881,"port_rcv_data":100398142614205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671852831704,"port_rcv_data":2390806632408,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99828064508095,"port_rcv_data":99170869196764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643386148536,"port_rcv_data":2288625156815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.465","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.478","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.492","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.504","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.516","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.527","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.537","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.547","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.3505485,"monotonic":2345969.782634852,"hcas":{"mlx5_0":{"port_xmit_data":98626852416922,"port_rcv_data":100398773538529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671853341034,"port_rcv_data":2390890645440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99828611622271,"port_rcv_data":99170945381040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643386351245,"port_rcv_data":2288689035364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.431","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.444","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.458","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.470","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:25.481","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:25.492","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:25.502","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:25.512","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.3513765,"monotonic":2345970.783463142,"hcas":{"mlx5_0":{"port_xmit_data":98626865028768,"port_rcv_data":100399404968885,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671853383458,"port_rcv_data":2390933674632,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99829201865428,"port_rcv_data":99171009902424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643386570493,"port_rcv_data":2288740906156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.452","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.467","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.57,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.480","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.492","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.504","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.514","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.525","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.534","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.3519516,"monotonic":2345971.784038064,"hcas":{"mlx5_0":{"port_xmit_data":98626877479879,"port_rcv_data":100399980784468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671853584663,"port_rcv_data":2390971285440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99829739984655,"port_rcv_data":99171077271896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643387103243,"port_rcv_data":2288796200164,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.469","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.483","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.15,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.496","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.508","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.01,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.519","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.541","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.76,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.550","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.3527396,"monotonic":2345972.784826314,"hcas":{"mlx5_0":{"port_xmit_data":98626890398819,"port_rcv_data":100400582679365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671854102682,"port_rcv_data":2391031434074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99830280770288,"port_rcv_data":99171131863040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643387293072,"port_rcv_data":2288838665598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.433","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.447","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.93,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.460","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.472","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.483","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.494","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.504","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.514","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.3535655,"monotonic":2345973.785651969,"hcas":{"mlx5_0":{"port_xmit_data":98626902898147,"port_rcv_data":100401161189825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671854312041,"port_rcv_data":2391070804258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99830820218162,"port_rcv_data":99171201263438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643387836269,"port_rcv_data":2288895720154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.434","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.449","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.27,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.461","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.473","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.484","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.495","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.505","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.514","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.354393,"monotonic":2345974.786479478,"hcas":{"mlx5_0":{"port_xmit_data":98626915696720,"port_rcv_data":100401770640603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671854869901,"port_rcv_data":2391143058954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99831359384455,"port_rcv_data":99171273906415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643388106682,"port_rcv_data":2288956152562,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.452","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.466","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.01,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.479","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.491","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.503","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.513","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.523","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.533","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.355201,"monotonic":2345975.787287536,"hcas":{"mlx5_0":{"port_xmit_data":98626928265645,"port_rcv_data":100402357598840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671855262501,"port_rcv_data":2391196848962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99831892770671,"port_rcv_data":99171342770685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643388618151,"port_rcv_data":2289012923130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.436","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.450","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.89,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.463","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.476","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.487","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.498","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.508","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.518","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.3560297,"monotonic":2345976.788116107,"hcas":{"mlx5_0":{"port_xmit_data":98626940906537,"port_rcv_data":100402958129099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671855652496,"port_rcv_data":2391260280156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99832430437521,"port_rcv_data":99171418250312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643388984556,"port_rcv_data":2289076387620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.457","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.471","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.484","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.496","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.507","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.517","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.528","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.537","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.356836,"monotonic":2345977.788922508,"hcas":{"mlx5_0":{"port_xmit_data":98626953601186,"port_rcv_data":100403589149891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671856041298,"port_rcv_data":2391351432679,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99832970376147,"port_rcv_data":99171521604792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643389387587,"port_rcv_data":2289167540975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.438","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.451","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.22,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.464","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.476","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.488","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.498","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.508","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:33.518","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.357664,"monotonic":2345978.789750567,"hcas":{"mlx5_0":{"port_xmit_data":98626966256264,"port_rcv_data":100404208158830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671856419994,"port_rcv_data":2391433649168,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99833505656612,"port_rcv_data":99171598261154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643389455058,"port_rcv_data":2289232041064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.438","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.452","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.464","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.476","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.487","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.498","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.508","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.518","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.3584962,"monotonic":2345979.790582656,"hcas":{"mlx5_0":{"port_xmit_data":98626978860295,"port_rcv_data":100404819393058,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671856763629,"port_rcv_data":2391506594989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99834044897519,"port_rcv_data":99171683390281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643389795423,"port_rcv_data":2289304988601,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.456","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.470","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.52,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.483","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.495","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.506","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.516","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.526","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.536","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.359304,"monotonic":2345980.791390614,"hcas":{"mlx5_0":{"port_xmit_data":98626991387383,"port_rcv_data":100405420890549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671856987759,"port_rcv_data":2391568367028,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99834584214661,"port_rcv_data":99171766254715,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643390201290,"port_rcv_data":2289375603200,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.440","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.454","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.85,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.468","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.480","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.491","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.502","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.513","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.522","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.36013,"monotonic":2345981.792216608,"hcas":{"mlx5_0":{"port_xmit_data":98627004234511,"port_rcv_data":100406081433595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671857337548,"port_rcv_data":2391624934306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99835189210738,"port_rcv_data":99171835212783,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643390551637,"port_rcv_data":2289432170478,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.460","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.474","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.487","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.498","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.510","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.520","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.530","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.540","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.3609414,"monotonic":2345982.793027615,"hcas":{"mlx5_0":{"port_xmit_data":98627017803077,"port_rcv_data":100406895505101,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671857702325,"port_rcv_data":2391699946781,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99835929779333,"port_rcv_data":99171931102487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643391032602,"port_rcv_data":2289514905465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.460","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.474","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.487","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.499","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.510","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.521","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.531","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.541","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.3617473,"monotonic":2345983.793833653,"hcas":{"mlx5_0":{"port_xmit_data":98627031571315,"port_rcv_data":100407705757171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671858250544,"port_rcv_data":2391770026906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99836668236334,"port_rcv_data":99172005159975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643391293467,"port_rcv_data":2289575797873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.460","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.474","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.61,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.486","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.499","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.16,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.510","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.520","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.530","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:39.540","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.3625557,"monotonic":2345984.794642199,"hcas":{"mlx5_0":{"port_xmit_data":98627045054164,"port_rcv_data":100408501552950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671858604037,"port_rcv_data":2391830329767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99837404225355,"port_rcv_data":99172079332209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643391791172,"port_rcv_data":2289636945902,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.444","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.457","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.59,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.470","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.93,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.483","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.494","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.506","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.516","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.526","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.3633797,"monotonic":2345985.795466211,"hcas":{"mlx5_0":{"port_xmit_data":98627058311741,"port_rcv_data":100409355195491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671859087002,"port_rcv_data":2391895697692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99838192816527,"port_rcv_data":99172149125939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643392146726,"port_rcv_data":2289694095716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.464","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.478","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.6,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.491","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.503","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.515","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.526","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.536","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.546","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.3641837,"monotonic":2345986.796270159,"hcas":{"mlx5_0":{"port_xmit_data":98627071006271,"port_rcv_data":100410299704561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671859360515,"port_rcv_data":2391956074864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99839079536177,"port_rcv_data":99172246988008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643392737199,"port_rcv_data":2289779503793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.464","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.478","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.92,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.491","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.503","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.514","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.525","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:42.535","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:42.546","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.364988,"monotonic":2345987.797074611,"hcas":{"mlx5_0":{"port_xmit_data":98627083671602,"port_rcv_data":100411262833677,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671859842100,"port_rcv_data":2392005343430,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99839993605964,"port_rcv_data":99172292172014,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643393109974,"port_rcv_data":2289812551196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.464","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.478","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.4,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.490","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.69,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.502","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.513","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.523","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.534","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.544","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.3658037,"monotonic":2345988.797890076,"hcas":{"mlx5_0":{"port_xmit_data":98627095812439,"port_rcv_data":100412407512698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671860307503,"port_rcv_data":2392170972642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99840970922121,"port_rcv_data":99172469283785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643393576570,"port_rcv_data":2289978215422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.464","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.478","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.06,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.491","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.503","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.515","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.525","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.536","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.545","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.3666108,"monotonic":2345989.798696976,"hcas":{"mlx5_0":{"port_xmit_data":98627107790592,"port_rcv_data":100413404697199,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671860615689,"port_rcv_data":2392188517697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99841952952967,"port_rcv_data":99172505157258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643393970673,"port_rcv_data":2290002701610,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.481","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.494","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.85,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.19,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.530","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:45.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:45.551","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:45.561","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.21,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.367405,"monotonic":2345990.79949145,"hcas":{"mlx5_0":{"port_xmit_data":98627119872251,"port_rcv_data":100414407349061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671861075652,"port_rcv_data":2392214627786,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99842927662143,"port_rcv_data":99172527208670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643394175025,"port_rcv_data":2290013030138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.448","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.462","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.92,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.475","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.487","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.498","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.508","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.519","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.528","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.3682325,"monotonic":2345991.800319016,"hcas":{"mlx5_0":{"port_xmit_data":98627131625915,"port_rcv_data":100415389510875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671861230094,"port_rcv_data":2392223676749,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99843903370518,"port_rcv_data":99172574928004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643394804697,"port_rcv_data":2290049225688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.484","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.498","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.22,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.511","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.27,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.523","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.534","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.545","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.555","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.565","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.3690233,"monotonic":2345992.801109796,"hcas":{"mlx5_0":{"port_xmit_data":98627143803014,"port_rcv_data":100416399732445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671861700038,"port_rcv_data":2392250822879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99844884159993,"port_rcv_data":99172595549728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643394957760,"port_rcv_data":2290058275168,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.468","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.482","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.03,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.495","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.57,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.507","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.519","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.540","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.550","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.3698301,"monotonic":2345993.801916578,"hcas":{"mlx5_0":{"port_xmit_data":98627155553988,"port_rcv_data":100417398926712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862011511,"port_rcv_data":2392268920460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99845867581883,"port_rcv_data":99172634054841,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433656,"port_rcv_data":2290085422099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.468","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.482","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.7,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.495","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.59,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.507","gpu_util_pct":99.0,"memory_used_mib":79963.0,"power_w":142.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.519","gpu_util_pct":98.0,"memory_used_mib":79915.0,"power_w":145.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.540","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.550","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.370639,"monotonic":2345994.802725585,"hcas":{"mlx5_0":{"port_xmit_data":98627167115431,"port_rcv_data":100418510592681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99846961687817,"port_rcv_data":99172645217649,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.468","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.482","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.99,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.495","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.62,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.507","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.519","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.529","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:50.540","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:50.549","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.3714516,"monotonic":2345995.803537942,"hcas":{"mlx5_0":{"port_xmit_data":98627178236845,"port_rcv_data":100419620532272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99848071492717,"port_rcv_data":99172656247249,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.471","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.485","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.57,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.499","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:51.511","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:51.522","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:51.533","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.543","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.554","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.3722625,"monotonic":2345996.804348794,"hcas":{"mlx5_0":{"port_xmit_data":98627189017025,"port_rcv_data":100420779707994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99849230312566,"port_rcv_data":99172666927803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.472","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.485","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.9,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.498","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.510","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.522","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.532","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.543","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.552","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.3730729,"monotonic":2345997.805159309,"hcas":{"mlx5_0":{"port_xmit_data":98627199688500,"port_rcv_data":100421947540288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99850397429234,"port_rcv_data":99172677488490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.471","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.485","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.97,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.498","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.18,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.510","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.521","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:53.532","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:53.543","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.553","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.3738818,"monotonic":2345998.805968529,"hcas":{"mlx5_0":{"port_xmit_data":98627210125910,"port_rcv_data":100423180641184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99851630653404,"port_rcv_data":99172687819978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.454","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.468","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.35,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.481","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.72,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.493","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.505","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.516","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.526","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.536","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.3747048,"monotonic":2345999.806791458,"hcas":{"mlx5_0":{"port_xmit_data":98627220463266,"port_rcv_data":100424398864039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99852849373796,"port_rcv_data":99172698036958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.455","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.469","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.45,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.482","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.82,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.494","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.505","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.515","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.525","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.535","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.3755312,"monotonic":2346000.80761794,"hcas":{"mlx5_0":{"port_xmit_data":98627230871456,"port_rcv_data":100425629897852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99854079615191,"port_rcv_data":99172708339193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.456","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.470","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.05,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.482","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.59,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:56.494","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:56.506","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:56.517","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:56.527","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:56.536","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.37594,"monotonic":2346001.808026543,"hcas":{"mlx5_0":{"port_xmit_data":98627241259489,"port_rcv_data":100426857031637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99855306224836,"port_rcv_data":99172718603223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.456","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.470","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.23,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.483","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.495","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.506","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.517","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.527","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.537","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.3767657,"monotonic":2346002.808852034,"hcas":{"mlx5_0":{"port_xmit_data":98627251509035,"port_rcv_data":100428066045602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99856513429847,"port_rcv_data":99172728730494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.476","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.489","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.35,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.503","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.81,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.515","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.526","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:58.537","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.547","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.557","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.3775709,"monotonic":2346003.809657311,"hcas":{"mlx5_0":{"port_xmit_data":98627261983352,"port_rcv_data":100429303674169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99857750920213,"port_rcv_data":99172739091254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.458","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.472","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.03,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.485","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.64,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.497","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.508","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.519","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.529","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.539","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.3783965,"monotonic":2346004.810482837,"hcas":{"mlx5_0":{"port_xmit_data":98627272340288,"port_rcv_data":100430527703983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99858974817370,"port_rcv_data":99172749343224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.476","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.490","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.55,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.503","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.9,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:00.514","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:00.525","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:00.536","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:00.546","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:00.556","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.3792067,"monotonic":2346005.811292796,"hcas":{"mlx5_0":{"port_xmit_data":98627282778603,"port_rcv_data":100431760923002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99860210144919,"port_rcv_data":99172759691688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.493","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.82,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.506","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.01,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.530","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.552","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.562","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.380011,"monotonic":2346006.812097484,"hcas":{"mlx5_0":{"port_xmit_data":98627293077951,"port_rcv_data":100432974967641,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862321061,"port_rcv_data":2392287017834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99861424558703,"port_rcv_data":99172769880451,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395433749,"port_rcv_data":2290085422530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.494","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.57,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:02.531","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:02.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.552","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.561","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.3808215,"monotonic":2346007.812907944,"hcas":{"mlx5_0":{"port_xmit_data":98627304012921,"port_rcv_data":100434142414518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862330764,"port_rcv_data":2392296590954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99862580339566,"port_rcv_data":99172790257432,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395443189,"port_rcv_data":2290094995650,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.494","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.83,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.49,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.530","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.551","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.561","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.3816288,"monotonic":2346008.813715224,"hcas":{"mlx5_0":{"port_xmit_data":98627315307692,"port_rcv_data":100435253840808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862344776,"port_rcv_data":2392310950634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99863676507746,"port_rcv_data":99172815794825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395457263,"port_rcv_data":2290109355330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.494","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.64,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.520","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.532","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.543","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.553","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.563","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.3824341,"monotonic":2346009.814520655,"hcas":{"mlx5_0":{"port_xmit_data":98627326758210,"port_rcv_data":100436394755391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862373110,"port_rcv_data":2392339669994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99864788662388,"port_rcv_data":99172855848302,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395485302,"port_rcv_data":2290138074690,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.493","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.71,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.86,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.531","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.542","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.552","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.562","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.383241,"monotonic":2346010.81532747,"hcas":{"mlx5_0":{"port_xmit_data":98627338050109,"port_rcv_data":100437505447341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862387138,"port_rcv_data":2392354029674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99865886298601,"port_rcv_data":99172881414533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395499330,"port_rcv_data":2290152434370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.494","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.02,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:06.530","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:06.540","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.551","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.561","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.3840508,"monotonic":2346011.816137536,"hcas":{"mlx5_0":{"port_xmit_data":98627349417022,"port_rcv_data":100438624191052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862401134,"port_rcv_data":2392368389354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99866991331329,"port_rcv_data":99172907046508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395513342,"port_rcv_data":2290166794050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.483","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.498","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.67,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.511","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.41,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.523","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.534","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.545","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.555","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.565","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.3848577,"monotonic":2346012.816944037,"hcas":{"mlx5_0":{"port_xmit_data":98627360716404,"port_rcv_data":100439736039096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862415487,"port_rcv_data":2392382749034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99868088758451,"port_rcv_data":99172932602633,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395527354,"port_rcv_data":2290181153730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.484","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.497","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.62,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.511","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.13,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.523","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.534","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.545","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.555","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.565","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.385665,"monotonic":2346013.817751429,"hcas":{"mlx5_0":{"port_xmit_data":98627372129122,"port_rcv_data":100440836609086,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862601813,"port_rcv_data":2392391590634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99869180671941,"port_rcv_data":99172952586384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643395695312,"port_rcv_data":2290189995330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.484","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.498","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.27,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.511","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.75,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.523","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.535","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.546","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.556","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.566","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259210.3864758,"monotonic":2346014.818562017,"hcas":{"mlx5_0":{"port_xmit_data":98627384365018,"port_rcv_data":100441841161566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671862819464,"port_rcv_data":2392443600040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99870131414049,"port_rcv_data":99173031256533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643396232681,"port_rcv_data":2290256867992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.484","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.498","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.73,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.512","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.524","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.535","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.545","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.556","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.565","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.3872721,"monotonic":2346015.81935868,"hcas":{"mlx5_0":{"port_xmit_data":98627397197736,"port_rcv_data":100442799220094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671863165718,"port_rcv_data":2392512477262,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99871020399765,"port_rcv_data":99173097701116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643396288342,"port_rcv_data":2290310965830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.468","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.481","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.9,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.494","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.507","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.518","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:11.529","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.539","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.548","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.3880877,"monotonic":2346016.820174215,"hcas":{"mlx5_0":{"port_xmit_data":98627409710610,"port_rcv_data":100443717664299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671863196548,"port_rcv_data":2392541933625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99871909364630,"port_rcv_data":99173139567545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643396318939,"port_rcv_data":2290340364929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.469","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.482","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.17,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.495","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.508","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.519","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.36,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.541","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.551","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.3889058,"monotonic":2346017.820992158,"hcas":{"mlx5_0":{"port_xmit_data":98627422129497,"port_rcv_data":100444629316705,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671863226943,"port_rcv_data":2392571361372,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99872793774288,"port_rcv_data":99173181341487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643396348745,"port_rcv_data":2290369792676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.505","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.522","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.59,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.535","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.547","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.558","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:13.569","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.579","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.589","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.389694,"monotonic":2346018.821780224,"hcas":{"mlx5_0":{"port_xmit_data":98627434619828,"port_rcv_data":100445539275648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671863402062,"port_rcv_data":2392601819877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99873674263562,"port_rcv_data":99173236541294,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643396677407,"port_rcv_data":2290412760319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.488","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.497","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.86,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.506","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.515","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.523","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.532","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.541","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.549","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.3905268,"monotonic":2346019.822613149,"hcas":{"mlx5_0":{"port_xmit_data":98627447619379,"port_rcv_data":100446473173762,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671863906339,"port_rcv_data":2392650116076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99874557119243,"port_rcv_data":99173284649329,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643397000071,"port_rcv_data":2290448548552,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.505","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.519","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.43,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:15.532","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:15.544","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:15.555","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.565","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.575","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.585","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.3913188,"monotonic":2346020.823405278,"hcas":{"mlx5_0":{"port_xmit_data":98627459664898,"port_rcv_data":100447403675169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671863915717,"port_rcv_data":2392659689196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99875478122508,"port_rcv_data":99173315013936,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643397201726,"port_rcv_data":2290466963272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.472","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.485","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.67,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.498","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.509","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.520","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.541","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.550","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.3921475,"monotonic":2346021.824233878,"hcas":{"mlx5_0":{"port_xmit_data":98627471597349,"port_rcv_data":100448417478654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671864275208,"port_rcv_data":2392701821990,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99876452659941,"port_rcv_data":99173371355900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643397602323,"port_rcv_data":2290511857626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.473","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.486","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.01,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.499","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.512","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.523","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.533","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.544","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.554","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.3929727,"monotonic":2346022.825059317,"hcas":{"mlx5_0":{"port_xmit_data":98627483642595,"port_rcv_data":100449454292352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671864639567,"port_rcv_data":2392757912216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99877432934923,"port_rcv_data":99173427518723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643397788478,"port_rcv_data":2290556344692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.492","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.506","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.12,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.518","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.530","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.541","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.551","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.561","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.571","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.393782,"monotonic":2346023.825868576,"hcas":{"mlx5_0":{"port_xmit_data":98627495310541,"port_rcv_data":100450472732542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671864684051,"port_rcv_data":2392801188962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99878405982090,"port_rcv_data":99173482204552,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643397831909,"port_rcv_data":2290599621438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.475","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.489","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.51,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.501","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.513","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.524","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.535","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.545","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.554","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.3946092,"monotonic":2346024.826695678,"hcas":{"mlx5_0":{"port_xmit_data":98627507076960,"port_rcv_data":100451500824319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671864728396,"port_rcv_data":2392844518290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99879390642237,"port_rcv_data":99173537179232,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643397875774,"port_rcv_data":2290642951938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.492","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.505","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.1,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.518","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.530","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:20.541","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.551","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.561","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.571","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.3954265,"monotonic":2346025.827512856,"hcas":{"mlx5_0":{"port_xmit_data":98627519407642,"port_rcv_data":100452493023333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671864815739,"port_rcv_data":2392931194404,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99880296135066,"port_rcv_data":99173636020806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643397963795,"port_rcv_data":2290729630524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.496","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.510","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.05,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.524","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.536","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.547","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.558","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.568","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.578","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.3962305,"monotonic":2346026.828316822,"hcas":{"mlx5_0":{"port_xmit_data":98627531918369,"port_rcv_data":100453411036520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671864845866,"port_rcv_data":2392960182810,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99881185123371,"port_rcv_data":99173680374992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643398022307,"port_rcv_data":2290761632630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.498","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.512","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.33,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.525","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.538","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:22.549","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.560","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.570","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.580","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.3970337,"monotonic":2346027.829120153,"hcas":{"mlx5_0":{"port_xmit_data":98627544800535,"port_rcv_data":100454331258775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671865199034,"port_rcv_data":2392988271122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99882080014408,"port_rcv_data":99173726721100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643398520601,"port_rcv_data":2290795548714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.513","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.527","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.51,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.540","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.552","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.563","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.573","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.583","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.593","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.3978252,"monotonic":2346028.829911756,"hcas":{"mlx5_0":{"port_xmit_data":98627557771507,"port_rcv_data":100455287989152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671865611706,"port_rcv_data":2393050243187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99882973106771,"port_rcv_data":99173808777938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643399036736,"port_rcv_data":2290865233017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.496","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.509","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.98,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.523","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.535","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.546","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.557","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.567","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.577","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.3986323,"monotonic":2346029.830718703,"hcas":{"mlx5_0":{"port_xmit_data":98627570628275,"port_rcv_data":100456196787608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671865952551,"port_rcv_data":2393067511864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99883866938652,"port_rcv_data":99173839320368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643399383533,"port_rcv_data":2290883217067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.493","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.4,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:25.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:25.530","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:25.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.552","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.562","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.3994544,"monotonic":2346030.83154111,"hcas":{"mlx5_0":{"port_xmit_data":98627583902932,"port_rcv_data":100457291356125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671866752180,"port_rcv_data":2393273646564,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99884755557279,"port_rcv_data":99174031679985,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643399718906,"port_rcv_data":2291063243536,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.480","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.494","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.07,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.507","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.519","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.531","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.551","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.561","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.4002786,"monotonic":2346031.832365186,"hcas":{"mlx5_0":{"port_xmit_data":98627595921313,"port_rcv_data":100458282430068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671866999328,"port_rcv_data":2393303540071,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99885716987285,"port_rcv_data":99174084237387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643400198724,"port_rcv_data":2291104128016,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.500","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:27.514","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.7,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:27.527","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.42,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:27.539","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.550","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.561","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.571","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.581","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.401084,"monotonic":2346032.833170342,"hcas":{"mlx5_0":{"port_xmit_data":98627608100358,"port_rcv_data":100459305381633,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671867405366,"port_rcv_data":2393337886956,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99886705789562,"port_rcv_data":99174128071447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643400520101,"port_rcv_data":2291136326114,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.500","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.514","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.4,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.527","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.540","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.552","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.562","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.573","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.583","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.4018888,"monotonic":2346033.833975223,"hcas":{"mlx5_0":{"port_xmit_data":98627620466017,"port_rcv_data":100460322217932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671867756689,"port_rcv_data":2393384444148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99887676738781,"port_rcv_data":99174195302061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643401014908,"port_rcv_data":2291191656418,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.517","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.82,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.543","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.556","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.567","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.578","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.588","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.598","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.4026775,"monotonic":2346034.834764098,"hcas":{"mlx5_0":{"port_xmit_data":98627634290801,"port_rcv_data":100461317034543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671868076361,"port_rcv_data":2393416763261,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99888637134218,"port_rcv_data":99174241214485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643401335758,"port_rcv_data":2291224183282,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.500","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.514","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.09,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.528","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.540","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.552","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.562","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.573","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.583","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.4034853,"monotonic":2346035.835571714,"hcas":{"mlx5_0":{"port_xmit_data":98627647164168,"port_rcv_data":100462364686883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671868398962,"port_rcv_data":2393449082236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99889654772069,"port_rcv_data":99174277045443,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643401505266,"port_rcv_data":2291247523152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.504","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.517","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.92,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.530","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.35,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.542","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.553","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.563","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.573","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.582","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.404293,"monotonic":2346036.836379611,"hcas":{"mlx5_0":{"port_xmit_data":98627659736175,"port_rcv_data":100463420339922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671868398962,"port_rcv_data":2393449082236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99890710568898,"port_rcv_data":99174307203804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643401854744,"port_rcv_data":2291265206352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.485","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.499","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.78,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.512","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.0,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.524","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.535","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.545","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.555","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.565","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.4051251,"monotonic":2346037.837211356,"hcas":{"mlx5_0":{"port_xmit_data":98627672343000,"port_rcv_data":100464607725298,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671868929620,"port_rcv_data":2393530203279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99891817159409,"port_rcv_data":99174382487663,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643402085183,"port_rcv_data":2291328645367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.487","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.500","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.19,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.513","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:33.525","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.536","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.547","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.557","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.567","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.4059517,"monotonic":2346038.838038223,"hcas":{"mlx5_0":{"port_xmit_data":98627685105475,"port_rcv_data":100465719393287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671869141567,"port_rcv_data":2393593148852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99892865827756,"port_rcv_data":99174457869344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643402294929,"port_rcv_data":2291391591452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.522","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.535","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.06,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.549","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.561","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.572","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.583","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.593","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.603","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.4067428,"monotonic":2346039.838829265,"hcas":{"mlx5_0":{"port_xmit_data":98627697663128,"port_rcv_data":100466787411445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671869155657,"port_rcv_data":2393607508670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99893919679493,"port_rcv_data":99174502184964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643402613841,"port_rcv_data":2291423529439,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.488","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.502","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.06,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:35.515","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.21,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:35.527","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.538","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.549","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.559","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.569","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.407569,"monotonic":2346040.839655524,"hcas":{"mlx5_0":{"port_xmit_data":98627710795109,"port_rcv_data":100467902727614,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671869780462,"port_rcv_data":2393671384127,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99894971356370,"port_rcv_data":99174552074103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643402790696,"port_rcv_data":2291461038815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.501","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.515","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.38,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.528","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.9,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.540","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.551","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.562","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.572","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.582","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.408382,"monotonic":2346041.840468427,"hcas":{"mlx5_0":{"port_xmit_data":98627723279250,"port_rcv_data":100468958127025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671869794737,"port_rcv_data":2393685743876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99896012278965,"port_rcv_data":99174587556069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643402955399,"port_rcv_data":2291484187648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.508","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.521","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.25,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:37.534","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.69,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:37.546","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.557","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.568","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.578","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.588","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.4091892,"monotonic":2346042.841275506,"hcas":{"mlx5_0":{"port_xmit_data":98627735906795,"port_rcv_data":100470030689419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671869808734,"port_rcv_data":2393700103556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99897070697258,"port_rcv_data":99174614426535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643402969365,"port_rcv_data":2291498547328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.508","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.522","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.06,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.536","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.98,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.548","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.559","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.569","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.580","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.590","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.4099958,"monotonic":2346043.842082253,"hcas":{"mlx5_0":{"port_xmit_data":98627748532417,"port_rcv_data":100471103339873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671869822746,"port_rcv_data":2393714463236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99898128885160,"port_rcv_data":99174641302297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643402983377,"port_rcv_data":2291512907008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.491","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.504","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.31,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.517","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.53,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.529","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.540","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.550","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.560","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.570","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.4108243,"monotonic":2346044.84291067,"hcas":{"mlx5_0":{"port_xmit_data":98627761085932,"port_rcv_data":100472163749734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671869832123,"port_rcv_data":2393724036356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99899177565771,"port_rcv_data":99174680977069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643403367761,"port_rcv_data":2291540163328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.511","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.525","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.75,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.538","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.49,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:40.551","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:40.562","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.572","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.583","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.593","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.4116292,"monotonic":2346045.843715806,"hcas":{"mlx5_0":{"port_xmit_data":98627774177257,"port_rcv_data":100473273072876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671870326651,"port_rcv_data":2393774950426,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99900238541491,"port_rcv_data":99174735950544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643403726447,"port_rcv_data":2291582670078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.528","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.542","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.9,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.555","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.69,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.568","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.579","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.590","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.600","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.611","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.4124162,"monotonic":2346046.844502828,"hcas":{"mlx5_0":{"port_xmit_data":98627786948661,"port_rcv_data":100474366593038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671870528678,"port_rcv_data":2393813633022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99901293154313,"port_rcv_data":99174777878808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643403757168,"port_rcv_data":2291612077966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.512","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.526","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.75,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.539","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.551","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.561","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.572","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.19,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.582","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.592","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.4132235,"monotonic":2346047.845310072,"hcas":{"mlx5_0":{"port_xmit_data":98627799460764,"port_rcv_data":100475469053972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671870588849,"port_rcv_data":2393872642840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99902335269130,"port_rcv_data":99174849155786,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643403817370,"port_rcv_data":2291671086484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.511","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.525","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.94,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.538","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.62,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.550","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.561","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.572","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.582","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.592","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.4140313,"monotonic":2346048.846118097,"hcas":{"mlx5_0":{"port_xmit_data":98627812033572,"port_rcv_data":100476551497540,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671870619461,"port_rcv_data":2393902186470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99903387198823,"port_rcv_data":99174891122879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643403847843,"port_rcv_data":2291700630178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.512","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.526","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.16,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.539","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.551","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":149.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.562","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.573","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.583","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.593","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.4148397,"monotonic":2346049.846926477,"hcas":{"mlx5_0":{"port_xmit_data":98627824614204,"port_rcv_data":100477633957997,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671870650787,"port_rcv_data":2393931722186,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99904441841677,"port_rcv_data":99174933137658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643403878641,"port_rcv_data":2291730166854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.537","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.550","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.77,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.563","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.72,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.576","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.587","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.598","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.608","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.618","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.4156246,"monotonic":2346050.84771131,"hcas":{"mlx5_0":{"port_xmit_data":98627837146956,"port_rcv_data":100478707966356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671870676796,"port_rcv_data":2393956471854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99905491052804,"port_rcv_data":99174981955157,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643404092495,"port_rcv_data":2291766561922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.516","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.530","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.87,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:46.543","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.555","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.567","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.578","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.588","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.598","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.41643,"monotonic":2346051.848516544,"hcas":{"mlx5_0":{"port_xmit_data":98627849971329,"port_rcv_data":100479802187503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671870949580,"port_rcv_data":2393996777846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99906544373135,"port_rcv_data":99175042621166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643404618363,"port_rcv_data":2291814915926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.516","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.529","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.98,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.543","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.555","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.566","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.577","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.587","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.597","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.4172368,"monotonic":2346052.849323307,"hcas":{"mlx5_0":{"port_xmit_data":98627862941347,"port_rcv_data":100480904448164,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671871364903,"port_rcv_data":2394045570029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99907597345164,"port_rcv_data":99175095759504,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643404817988,"port_rcv_data":2291855611785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.514","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.525","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.74,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.535","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.13,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.545","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.556","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.566","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.575","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.584","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.41806,"monotonic":2346053.850146485,"hcas":{"mlx5_0":{"port_xmit_data":98627875721575,"port_rcv_data":100481995073201,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671871699176,"port_rcv_data":2394092629267,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99908643074259,"port_rcv_data":99175152588178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643405146960,"port_rcv_data":2291900055384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.515","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.37,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.529","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.66,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.542","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.48,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.554","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.565","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.576","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.586","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.595","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.62,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.4188716,"monotonic":2346054.850958163,"hcas":{"mlx5_0":{"port_xmit_data":98627888601081,"port_rcv_data":100483111199682,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671872054126,"port_rcv_data":2394160094104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99909694119619,"port_rcv_data":99175232389843,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643405358178,"port_rcv_data":2291967442372,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.520","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.534","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.9,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.547","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.4,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.559","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.570","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.581","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.592","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.602","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.419676,"monotonic":2346055.851762685,"hcas":{"mlx5_0":{"port_xmit_data":98627900953835,"port_rcv_data":100484230026098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671872094178,"port_rcv_data":2394198198455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99910770464040,"port_rcv_data":99175273764791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643405388946,"port_rcv_data":2291996645979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.520","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.534","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.88,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.547","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:51.560","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.571","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.582","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.592","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.602","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.4204822,"monotonic":2346056.852568577,"hcas":{"mlx5_0":{"port_xmit_data":98627912867011,"port_rcv_data":100485398546482,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671872110391,"port_rcv_data":2394213042158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99911926013685,"port_rcv_data":99175308159103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643405534723,"port_rcv_data":2292019221162,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:52.522","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":166.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:52.536","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.5,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:52.549","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.18,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.561","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.572","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":149.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.583","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.593","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.603","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.4212863,"monotonic":2346057.853372925,"hcas":{"mlx5_0":{"port_xmit_data":98627925098422,"port_rcv_data":100486644533139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671872358453,"port_rcv_data":2394293368126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99913092893519,"port_rcv_data":99175419099037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643406153359,"port_rcv_data":2292118374474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.520","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.533","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.28,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.546","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.558","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.568","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":149.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.579","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.589","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.598","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.4220958,"monotonic":2346058.854182405,"hcas":{"mlx5_0":{"port_xmit_data":98627937685309,"port_rcv_data":100487877690190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671872852050,"port_rcv_data":2394350510420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99914266465198,"port_rcv_data":99175470433934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643406338646,"port_rcv_data":2292157715894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.519","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.533","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.9,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.546","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.558","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.568","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.579","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.589","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.599","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.4229062,"monotonic":2346059.854992554,"hcas":{"mlx5_0":{"port_xmit_data":98627950200960,"port_rcv_data":100488965026223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671873015420,"port_rcv_data":2394368872754,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99915335813261,"port_rcv_data":99175518531504,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643406796857,"port_rcv_data":2292193656690,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.524","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.538","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.32,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.550","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.07,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.562","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.573","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.584","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.594","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.603","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.4237125,"monotonic":2346060.855799147,"hcas":{"mlx5_0":{"port_xmit_data":98627963161082,"port_rcv_data":100490053713276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671873488666,"port_rcv_data":2394409599524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99916386080523,"port_rcv_data":99175554363095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643406964288,"port_rcv_data":2292217065850,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.524","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.537","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.44,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:56.550","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.21,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.563","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.574","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.585","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.595","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.605","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.4239385,"monotonic":2346061.85602524,"hcas":{"mlx5_0":{"port_xmit_data":98627975986138,"port_rcv_data":100491143430548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671873798790,"port_rcv_data":2394441457658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99917442218127,"port_rcv_data":99175608252847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643407454987,"port_rcv_data":2292258572246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.523","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.537","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":142.66,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.550","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.562","gpu_util_pct":98.0,"memory_used_mib":79963.0,"power_w":143.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.573","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":148.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.585","gpu_util_pct":99.0,"memory_used_mib":79959.0,"power_w":148.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.595","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":146.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.604","gpu_util_pct":98.0,"memory_used_mib":80045.0,"power_w":144.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.424757,"monotonic":2346062.856843593,"hcas":{"mlx5_0":{"port_xmit_data":98627988330197,"port_rcv_data":100492355293275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671874152810,"port_rcv_data":2394488873519,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99918607134158,"port_rcv_data":99175648870835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643407483461,"port_rcv_data":2292287292253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.524","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.538","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.12,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.551","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.563","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.574","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.586","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.595","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.605","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.425567,"monotonic":2346063.857653428,"hcas":{"mlx5_0":{"port_xmit_data":98628000308129,"port_rcv_data":100493538077780,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671874166776,"port_rcv_data":2394503233199,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99919777168642,"port_rcv_data":99175675115535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643407497426,"port_rcv_data":2292301651933,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.524","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.538","gpu_util_pct":97.0,"memory_used_mib":79959.0,"power_w":148.16,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.551","gpu_util_pct":97.0,"memory_used_mib":79943.0,"power_w":149.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.563","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.575","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":148.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.586","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":144.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.596","gpu_util_pct":97.0,"memory_used_mib":79943.0,"power_w":146.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.606","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.4263723,"monotonic":2346064.858458924,"hcas":{"mlx5_0":{"port_xmit_data":98628012500738,"port_rcv_data":100494728732398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671874180788,"port_rcv_data":2394517592879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99920950954114,"port_rcv_data":99175701517143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643407511423,"port_rcv_data":2292316011613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:00.541","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:00.555","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.27,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.568","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.67,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.580","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.591","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.602","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.612","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.622","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.4271634,"monotonic":2346065.85924986,"hcas":{"mlx5_0":{"port_xmit_data":98628024553705,"port_rcv_data":100495900394987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671874180788,"port_rcv_data":2394517592879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99922125496574,"port_rcv_data":99175731183020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643407855430,"port_rcv_data":2292333694813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.528","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.31,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.555","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.01,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.567","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.578","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.589","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.599","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.609","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.4279747,"monotonic":2346066.860060993,"hcas":{"mlx5_0":{"port_xmit_data":98628037294474,"port_rcv_data":100497211346241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671874762239,"port_rcv_data":2394644723919,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99923309527960,"port_rcv_data":99175852605340,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408137840,"port_rcv_data":2292443144337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.527","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.59,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.554","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.567","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.578","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.589","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.599","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.609","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.4287813,"monotonic":2346067.860867737,"hcas":{"mlx5_0":{"port_xmit_data":98628049492138,"port_rcv_data":100498388629455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671874918479,"port_rcv_data":2394653565726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99924478747583,"port_rcv_data":99175900083904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408748400,"port_rcv_data":2292478742917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.528","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.74,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.554","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.74,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.565","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.576","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.587","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.597","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.606","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.429591,"monotonic":2346068.861677558,"hcas":{"mlx5_0":{"port_xmit_data":98628062254066,"port_rcv_data":100499606605360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526358,"port_rcv_data":2394689241114,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99925660429015,"port_rcv_data":99175912129691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408748586,"port_rcv_data":2292478743779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.511","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.524","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.63,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.537","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.9,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.549","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.560","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.571","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.581","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.591","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.4304168,"monotonic":2346069.862503446,"hcas":{"mlx5_0":{"port_xmit_data":98628074324635,"port_rcv_data":100500779136860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99926832685681,"port_rcv_data":99175932992618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.511","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.525","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.42,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:05.538","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.69,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.550","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.561","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.572","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.582","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.592","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.4312396,"monotonic":2346070.863326165,"hcas":{"mlx5_0":{"port_xmit_data":98628085756822,"port_rcv_data":100501989410533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99928041274357,"port_rcv_data":99175944304273,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.532","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.545","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.63,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.558","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.59,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.570","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.582","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.593","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.604","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.613","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.4320416,"monotonic":2346071.864128339,"hcas":{"mlx5_0":{"port_xmit_data":98628097137192,"port_rcv_data":100503213018756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99929266347014,"port_rcv_data":99175955589092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.513","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.527","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.08,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.539","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.551","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.562","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.572","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.582","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.592","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.4328687,"monotonic":2346072.864955237,"hcas":{"mlx5_0":{"port_xmit_data":98628108604757,"port_rcv_data":100504446240260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99930499976711,"port_rcv_data":99175966950859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:08.536","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:08.550","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.81,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.563","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.575","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.587","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.597","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.608","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.617","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.4336715,"monotonic":2346073.86575803,"hcas":{"mlx5_0":{"port_xmit_data":98628120473831,"port_rcv_data":100505645353231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99931696311192,"port_rcv_data":99175978684282,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.537","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.550","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.15,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.563","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.52,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.575","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.586","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.596","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.606","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.616","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.4344761,"monotonic":2346074.866562569,"hcas":{"mlx5_0":{"port_xmit_data":98628132488690,"port_rcv_data":100506814066810,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99932867346391,"port_rcv_data":99175990634789,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.537","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.551","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.06,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.564","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.576","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.587","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.597","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.608","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.617","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.4352787,"monotonic":2346075.867365095,"hcas":{"mlx5_0":{"port_xmit_data":98628144491895,"port_rcv_data":100507980893156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99934034165523,"port_rcv_data":99176002540125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.540","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:11.554","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.33,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.567","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.43,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.579","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.591","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.601","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.42,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.612","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.622","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.4360776,"monotonic":2346076.868164125,"hcas":{"mlx5_0":{"port_xmit_data":98628156580803,"port_rcv_data":100509153676797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99935204992844,"port_rcv_data":99176014507513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:12.539","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":157.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.553","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.57,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.566","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.579","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.590","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.601","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.45,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.611","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.621","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.11,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.4368784,"monotonic":2346077.868964747,"hcas":{"mlx5_0":{"port_xmit_data":98628168644535,"port_rcv_data":100510326397793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99936379432178,"port_rcv_data":99176026493270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.535","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.549","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.55,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.562","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.574","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.585","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.595","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.605","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.615","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.4376874,"monotonic":2346078.869773806,"hcas":{"mlx5_0":{"port_xmit_data":98628180733441,"port_rcv_data":100511502917975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99937555812189,"port_rcv_data":99176038485819,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.541","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.555","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.09,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.567","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.579","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.590","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.601","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.611","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.621","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.4384906,"monotonic":2346079.870577243,"hcas":{"mlx5_0":{"port_xmit_data":98628192766813,"port_rcv_data":100512671639931,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99938724530884,"port_rcv_data":99176050418651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.540","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.553","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.91,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.566","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.15,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.578","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.589","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.600","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.610","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.620","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.4392965,"monotonic":2346080.871383054,"hcas":{"mlx5_0":{"port_xmit_data":98628204764230,"port_rcv_data":100513842208342,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99939894556790,"port_rcv_data":99176062324371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:16.544","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:16.557","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.73,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:16.570","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.5,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.583","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.594","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.605","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.615","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.625","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.4400973,"monotonic":2346081.872183704,"hcas":{"mlx5_0":{"port_xmit_data":98628216791831,"port_rcv_data":100515008154961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99941058066864,"port_rcv_data":99176074209525,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.521","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.535","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.98,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.547","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.05,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.559","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.570","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":147.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.581","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.591","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.601","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":148.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.4409237,"monotonic":2346082.873010139,"hcas":{"mlx5_0":{"port_xmit_data":98628228774208,"port_rcv_data":100516172517422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99942222293018,"port_rcv_data":99176086089446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:18.521","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":166.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:18.535","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.38,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.548","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.92,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.559","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.570","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":149.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.580","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.591","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.600","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.441751,"monotonic":2346083.87383753,"hcas":{"mlx5_0":{"port_xmit_data":98628241209726,"port_rcv_data":100517223260442,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99943273014936,"port_rcv_data":99176098431119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.544","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.558","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.97,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.571","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.583","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.594","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.605","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.27,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.615","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.625","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.4425535,"monotonic":2346084.874640203,"hcas":{"mlx5_0":{"port_xmit_data":98628254246175,"port_rcv_data":100518216896455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99944267486940,"port_rcv_data":99176111394923,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.545","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":165.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.558","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.32,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.571","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.582","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.593","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.604","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.614","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.624","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.443359,"monotonic":2346085.875445535,"hcas":{"mlx5_0":{"port_xmit_data":98628267736296,"port_rcv_data":100519177394513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99945228153785,"port_rcv_data":99176124803771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.548","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.562","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.19,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.575","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.587","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.598","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.608","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.619","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.629","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.4441605,"monotonic":2346086.876247107,"hcas":{"mlx5_0":{"port_xmit_data":98628281171651,"port_rcv_data":100520133938068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99946185389242,"port_rcv_data":99176138166520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:22.527","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:22.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.47,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.554","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.566","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.577","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.588","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.598","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.608","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.444985,"monotonic":2346087.877071233,"hcas":{"mlx5_0":{"port_xmit_data":98628294586752,"port_rcv_data":100521088910777,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99947143121985,"port_rcv_data":99176151479249,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.548","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.561","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.34,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.574","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.587","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.598","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.609","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.620","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.630","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.445787,"monotonic":2346088.877873215,"hcas":{"mlx5_0":{"port_xmit_data":98628308091212,"port_rcv_data":100522050629806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99948101861468,"port_rcv_data":99176164919985,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.526","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.540","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.34,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.553","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.38,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.565","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.576","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.587","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.597","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.607","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.4466121,"monotonic":2346089.878698679,"hcas":{"mlx5_0":{"port_xmit_data":98628321568675,"port_rcv_data":100523009559232,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99949058433794,"port_rcv_data":99176178263755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.527","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.541","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.28,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.554","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.566","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.577","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.587","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.598","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.607","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.4474442,"monotonic":2346090.879530624,"hcas":{"mlx5_0":{"port_xmit_data":98628334988284,"port_rcv_data":100523964937969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99950015927479,"port_rcv_data":99176191636328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.573","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.587","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.98,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.600","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.612","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.623","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.634","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.73,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.644","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.654","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.448225,"monotonic":2346091.880311381,"hcas":{"mlx5_0":{"port_xmit_data":98628348464703,"port_rcv_data":100524924180885,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99950972810034,"port_rcv_data":99176204991049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:27.529","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.543","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.45,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.556","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.89,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.569","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.580","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.590","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.601","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.611","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.4490538,"monotonic":2346092.881140101,"hcas":{"mlx5_0":{"port_xmit_data":98628362277712,"port_rcv_data":100525809013135,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99951859282304,"port_rcv_data":99176218772511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.573","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":160.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.587","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.61,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.600","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.612","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.623","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.633","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.643","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.653","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.4498377,"monotonic":2346093.881924106,"hcas":{"mlx5_0":{"port_xmit_data":98628376602306,"port_rcv_data":100526611168888,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99952661348000,"port_rcv_data":99176233040601,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.552","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":161.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.566","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.42,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.579","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.591","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.603","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.614","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.624","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.634","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.4506402,"monotonic":2346094.882726514,"hcas":{"mlx5_0":{"port_xmit_data":98628390934537,"port_rcv_data":100527413895018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99953462203251,"port_rcv_data":99176247270012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:30.553","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":163.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:30.567","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.07,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.580","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.592","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.603","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.614","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.624","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.634","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.4514415,"monotonic":2346095.883527922,"hcas":{"mlx5_0":{"port_xmit_data":98628405261710,"port_rcv_data":100528216726079,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99954267158502,"port_rcv_data":99176261582988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.553","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.567","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.27,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.580","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.593","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.604","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.615","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.626","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.636","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.4515843,"monotonic":2346096.883670598,"hcas":{"mlx5_0":{"port_xmit_data":98628419537921,"port_rcv_data":100529015999952,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99955064096733,"port_rcv_data":99176275754122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.556","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":162.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.569","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.65,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.582","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.594","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.83,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.605","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.616","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.626","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.34,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.636","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.4523928,"monotonic":2346097.884478674,"hcas":{"mlx5_0":{"port_xmit_data":98628433565698,"port_rcv_data":100529752893817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99955801935279,"port_rcv_data":99176289742402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.533","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.547","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.98,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.561","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.09,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.573","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.580","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.589","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.598","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.607","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.453227,"monotonic":2346098.885313445,"hcas":{"mlx5_0":{"port_xmit_data":98628447058876,"port_rcv_data":100530344951962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99956394424867,"port_rcv_data":99176303185166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.534","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.548","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.27,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.562","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.574","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.586","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.596","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.606","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.616","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.4540498,"monotonic":2346099.886136212,"hcas":{"mlx5_0":{"port_xmit_data":98628460542955,"port_rcv_data":100530936089854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99956983852636,"port_rcv_data":99176316546104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.535","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.548","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.561","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.573","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.585","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.595","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.605","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.615","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.4548757,"monotonic":2346100.886962135,"hcas":{"mlx5_0":{"port_xmit_data":98628473980181,"port_rcv_data":100531525402656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99957574199586,"port_rcv_data":99176329939933,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.552","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.566","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.33,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.579","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.591","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.602","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.613","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.623","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.632","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.455686,"monotonic":2346101.887772849,"hcas":{"mlx5_0":{"port_xmit_data":98628486031821,"port_rcv_data":100532059017101,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99958107460537,"port_rcv_data":99176341917336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.556","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.570","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.86,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.583","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.594","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.605","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.615","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.626","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.635","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.4564946,"monotonic":2346102.888580996,"hcas":{"mlx5_0":{"port_xmit_data":98628493059644,"port_rcv_data":100532381491781,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99958429880764,"port_rcv_data":99176348876733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.536","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.549","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":125.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.562","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.574","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.586","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.596","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":125.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.606","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.616","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":124.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.4573252,"monotonic":2346103.889411768,"hcas":{"mlx5_0":{"port_xmit_data":98628497339211,"port_rcv_data":100532579303903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99958626712568,"port_rcv_data":99176353090217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.579","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":120.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.594","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":107.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.608","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":108.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.621","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":109.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.634","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":107.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.648","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":102.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.662","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":102.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:39.675","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":99.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.458091,"monotonic":2346104.89017735,"hcas":{"mlx5_0":{"port_xmit_data":98628497339211,"port_rcv_data":100532579303903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99958626712568,"port_rcv_data":99176353090217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.602","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":85.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.619","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.635","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.650","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":81.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.665","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":90.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.679","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":84.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:40.692","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":83.45,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.705","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":81.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.4588242,"monotonic":2346105.890910642,"hcas":{"mlx5_0":{"port_xmit_data":98628497339211,"port_rcv_data":100532579303903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99958626712568,"port_rcv_data":99176353090217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.606","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":84.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.622","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":79.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.636","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":79.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.650","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":81.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.664","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":83.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.678","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.692","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.705","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259302.4595597,"monotonic":2346106.89164598,"hcas":{"mlx5_0":{"port_xmit_data":98628497339211,"port_rcv_data":100532579303903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99958626712568,"port_rcv_data":99176353090217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:42.602","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":83.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:42.619","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":79.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:42.634","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":79.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:42.646","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":81.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:42.659","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":83.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:42.671","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:42.683","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:42.695","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259303.460307,"monotonic":2346107.892393472,"hcas":{"mlx5_0":{"port_xmit_data":98628497339211,"port_rcv_data":100532579303903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2671875526389,"port_rcv_data":2394689241183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":99958626712568,"port_rcv_data":99176353090217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2643408901835,"port_rcv_data":2292487662773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:43.602","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":83.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:43.617","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":79.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:43.631","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":79.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:43.645","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":81.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:43.659","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":83.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:43.673","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:43.686","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:43.699","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_7.jsonl

@@ -0,0 +1,459 @@
+{"wall_time":1788258845.428376,"monotonic":2345651.391285991,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.579","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.604","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.621","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.54,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.636","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.651","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.665","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.679","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.692","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.4289222,"monotonic":2345652.391832414,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.573","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.591","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.608","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.623","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.637","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.651","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.664","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.677","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.4291131,"monotonic":2345653.392023348,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.553","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.86,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.571","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.587","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.602","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.616","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.629","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.642","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.655","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.429875,"monotonic":2345654.392785294,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.573","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.590","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.607","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.622","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.636","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.649","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.661","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.674","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.4306147,"monotonic":2345655.393525101,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.553","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.572","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.589","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.605","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.619","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.634","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.647","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.660","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.4313684,"monotonic":2345656.394278799,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.556","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.573","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.590","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.606","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.620","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.634","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.647","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.660","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.432126,"monotonic":2345657.395035993,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.581","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.599","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.616","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.631","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.646","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.659","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.673","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.685","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.4328558,"monotonic":2345658.395766126,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.554","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.572","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.589","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.605","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.620","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.634","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.647","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.660","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.4331136,"monotonic":2345659.396024057,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.581","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.599","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.616","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.631","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.646","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.659","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.673","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.686","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.4338477,"monotonic":2345660.39675779,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.558","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.577","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.594","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.610","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.626","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.640","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.654","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.668","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.434217,"monotonic":2345661.397126822,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.556","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.574","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.591","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.607","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.622","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.636","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.650","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.663","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.4349709,"monotonic":2345662.39788123,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.557","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.574","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.591","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.607","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.624","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.638","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.652","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.666","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.4357197,"monotonic":2345663.398630011,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.580","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.599","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.616","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.631","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.646","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.661","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.674","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.687","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.4364486,"monotonic":2345664.399358778,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.560","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.578","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.595","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.610","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.625","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.639","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.652","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.665","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.437203,"monotonic":2345665.400113143,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.541","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.559","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.576","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.592","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.607","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.621","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.635","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.648","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.4379728,"monotonic":2345666.400883039,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.562","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.581","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.599","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.615","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.630","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.645","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.659","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.672","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.438721,"monotonic":2345667.401631426,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.584","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.603","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.619","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.635","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.649","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.663","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.682","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.696","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.439444,"monotonic":2345668.402354529,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.565","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.584","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.601","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.617","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.632","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.646","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.660","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.673","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.4401908,"monotonic":2345669.403101031,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.565","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.583","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.599","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.615","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.630","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.643","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.657","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.669","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.440948,"monotonic":2345670.403858389,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.565","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.584","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.601","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.618","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.633","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.648","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.662","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.675","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.4411132,"monotonic":2345671.404023868,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.564","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.580","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.592","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.604","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.615","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.627","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.638","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.650","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.441889,"monotonic":2345672.404799631,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.584","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.601","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.618","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.634","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.648","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.662","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.675","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.688","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.4426363,"monotonic":2345673.405546636,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.564","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.581","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.598","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.613","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.627","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.641","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.654","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.667","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.4433968,"monotonic":2345674.406307414,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.542","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.560","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.576","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.591","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.605","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.619","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.632","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.644","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.4442024,"monotonic":2345675.407112806,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.589","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.608","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.625","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.640","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.656","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.670","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.683","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.697","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.4449317,"monotonic":2345676.407842402,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.549","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.567","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.585","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.601","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.616","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.631","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.645","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.658","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.4457092,"monotonic":2345677.408619448,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.565","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.583","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.600","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.616","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.631","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.645","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.659","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.672","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.4464607,"monotonic":2345678.409371215,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.560","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.578","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.595","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.610","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.625","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.639","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.652","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.665","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.4472394,"monotonic":2345679.4101496,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.568","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.587","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.604","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.620","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.635","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.649","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.663","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.677","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.4479938,"monotonic":2345680.410904125,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.596","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.614","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.631","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.646","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.660","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.674","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.687","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.700","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.4487278,"monotonic":2345681.411638191,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.572","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.589","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.605","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.620","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.634","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.648","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.661","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.673","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.4493687,"monotonic":2345682.412278884,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.575","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.598","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.614","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.629","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.643","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.657","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.670","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.683","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.4501202,"monotonic":2345683.413029887,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.596","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.613","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.630","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.35,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.645","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.660","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.674","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.687","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:37.700","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.4508483,"monotonic":2345684.413758604,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.576","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.594","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.610","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.625","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.640","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.654","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.667","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.679","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.4516058,"monotonic":2345685.414515303,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.571","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.589","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.606","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.621","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.636","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.650","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.663","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.676","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.4523606,"monotonic":2345686.415271008,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.596","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.614","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.630","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.646","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.660","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.674","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.687","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.700","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.453155,"monotonic":2345687.416065223,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.571","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.36,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.589","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.605","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.08,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.621","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.635","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.648","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.661","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:41.674","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.4539142,"monotonic":2345688.416824435,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.595","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.613","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.629","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.644","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.658","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.672","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.685","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.698","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.4546504,"monotonic":2345689.417560744,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.596","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.614","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.630","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.646","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.660","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.674","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.688","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.701","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.4553854,"monotonic":2345690.418295812,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.575","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.592","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.609","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.623","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.637","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.651","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.664","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.677","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.4561467,"monotonic":2345691.419057242,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.603","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.620","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.637","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.652","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.667","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.681","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.694","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.707","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.4568791,"monotonic":2345692.419789547,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.603","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.622","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.638","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.653","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.667","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.681","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.696","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.709","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.457118,"monotonic":2345693.420028245,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.579","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.597","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.613","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.628","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":80.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.642","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.656","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.669","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:47.682","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.4574578,"monotonic":2345694.420368163,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.554","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.94,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.571","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.587","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.99,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.602","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":80.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.616","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.630","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.643","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.656","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.4582438,"monotonic":2345695.421154187,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.559","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.576","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.592","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.607","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.622","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.635","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.648","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.7,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.661","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.4590333,"monotonic":2345696.421942165,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.539","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.556","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.573","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.588","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.603","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.616","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.629","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.642","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.4598331,"monotonic":2345697.422743418,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.607","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.625","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.641","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.656","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.672","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.687","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.701","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:51.714","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.4605675,"monotonic":2345698.423476114,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.585","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.603","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.619","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.634","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.649","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.663","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.676","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.689","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.4613233,"monotonic":2345699.424233821,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.600","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.618","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.635","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.650","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.664","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.678","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.692","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.68,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.705","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.4620636,"monotonic":2345700.424973973,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.587","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.09,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.605","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.621","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.635","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.649","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.663","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.676","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.689","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.4628193,"monotonic":2345701.425729735,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.607","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.625","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.642","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.657","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.672","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.686","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.699","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.712","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.4635515,"monotonic":2345702.426461846,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.609","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.627","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.644","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.660","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.675","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.690","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.704","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.718","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.4642727,"monotonic":2345703.427182815,"hcas":{"mlx5_0":{"port_xmit_data":146690530091532,"port_rcv_data":148621672522485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272534435,"port_rcv_data":2411418042150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393886785,"port_rcv_data":147221946365393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216149348,"port_rcv_data":2286038592711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.543","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.562","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.578","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.12,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.595","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.610","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.624","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.638","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.652","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.4650512,"monotonic":2345704.42796161,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.569","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.587","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.605","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.621","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.636","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.651","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.665","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.679","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.4658012,"monotonic":2345705.428711714,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.611","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.630","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.647","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.662","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.677","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.692","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.706","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.719","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.466524,"monotonic":2345706.429434412,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.607","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.625","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.643","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.658","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.673","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.688","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.702","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.715","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.4672444,"monotonic":2345707.430154821,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.588","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.606","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.623","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":80.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.639","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.654","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.669","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.683","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.697","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.4679823,"monotonic":2345708.430892813,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.570","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.589","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.606","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.622","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.637","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.652","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.666","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.680","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.4687383,"monotonic":2345709.431648976,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.583","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.602","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.619","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.635","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.650","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.665","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.679","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.692","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.4694915,"monotonic":2345710.432402196,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.570","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.589","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.606","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.621","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.636","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.650","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.664","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.678","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.470249,"monotonic":2345711.433159665,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.570","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.589","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.606","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.622","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.637","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.651","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.665","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.679","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.4710066,"monotonic":2345712.433917233,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.591","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.610","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.626","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.642","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.657","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.671","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.685","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.699","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.4717453,"monotonic":2345713.434655789,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.575","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.593","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.609","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.19,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.625","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.640","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.654","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.668","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.681","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.4725115,"monotonic":2345714.435421387,"hcas":{"mlx5_0":{"port_xmit_data":146690530114937,"port_rcv_data":148621696455285,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272557825,"port_rcv_data":2411441974950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393910453,"port_rcv_data":147221970298193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216172954,"port_rcv_data":2286062525511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.574","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.593","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.610","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.17,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.626","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.641","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.655","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.669","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.683","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.4732685,"monotonic":2345715.436179129,"hcas":{"mlx5_0":{"port_xmit_data":146690530128949,"port_rcv_data":148621710814965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272571837,"port_rcv_data":2411456334630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393924481,"port_rcv_data":147221984657873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216186966,"port_rcv_data":2286076885191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.621","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.639","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.656","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.671","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.686","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.700","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.714","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.727","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.4739869,"monotonic":2345716.436897402,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.596","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.614","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.631","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.646","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.661","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.675","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.689","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.702","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.4747288,"monotonic":2345717.437639297,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.575","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.593","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.609","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.625","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.640","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.654","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:11.668","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:11.681","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.4754999,"monotonic":2345718.438409577,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.592","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.610","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.626","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.642","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.656","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.670","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.683","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.696","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.4762619,"monotonic":2345719.439172293,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.596","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.614","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.631","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.646","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.661","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.675","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.689","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.702","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.4770162,"monotonic":2345720.439926843,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.600","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.618","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.634","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.650","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.664","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.678","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.691","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.704","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.4777706,"monotonic":2345721.440680913,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.597","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.615","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.632","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.647","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.662","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.676","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.690","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:15.704","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.4785259,"monotonic":2345722.441436265,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.579","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.596","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.613","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.630","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.645","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.660","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.674","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.687","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.479296,"monotonic":2345723.442206438,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.619","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.637","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.653","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.668","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.683","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.697","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.710","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.723","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.4800365,"monotonic":2345724.442946994,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.582","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.600","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.616","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.632","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.646","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.660","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.673","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.686","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.4808137,"monotonic":2345725.443724126,"hcas":{"mlx5_0":{"port_xmit_data":146690530142930,"port_rcv_data":148621725174645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272585864,"port_rcv_data":2411470694310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393938462,"port_rcv_data":147221999017553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216200978,"port_rcv_data":2286091244871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.623","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.640","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.657","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.672","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.687","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:19.701","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:19.714","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:19.727","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.4815605,"monotonic":2345726.444471059,"hcas":{"mlx5_0":{"port_xmit_data":146690530156911,"port_rcv_data":148621739534325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272599876,"port_rcv_data":2411485053990,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393952505,"port_rcv_data":147222013377233,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216214990,"port_rcv_data":2286105604551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.579","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.597","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.613","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.628","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.643","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.656","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.670","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.683","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.482355,"monotonic":2345727.445265363,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.585","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.603","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.619","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.635","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.650","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.664","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:21.679","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:21.695","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.483115,"monotonic":2345728.446025382,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.600","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.617","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.634","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.650","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.664","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:22.678","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.692","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.705","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.4838796,"monotonic":2345729.4467899,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.603","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.621","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.638","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.654","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.668","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.682","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.696","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.709","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.4846404,"monotonic":2345730.447550611,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.611","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.629","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.645","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.661","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.676","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.690","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:24.703","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:24.717","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.4854004,"monotonic":2345731.448310323,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.564","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.582","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.599","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.614","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.629","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.643","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.655","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.668","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":79.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.4862094,"monotonic":2345732.449119542,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.587","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.605","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.621","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.636","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.651","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.665","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.678","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.691","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.4869907,"monotonic":2345733.44990111,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.587","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.604","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.621","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.636","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.650","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.664","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.677","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:27.690","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.4877772,"monotonic":2345734.450687459,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.611","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":81.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.628","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.645","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.660","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.675","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.689","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.702","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.716","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.4885345,"monotonic":2345735.451444679,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.591","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.608","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.625","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.640","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.654","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.668","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:29.681","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:29.694","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.4893103,"monotonic":2345736.452220563,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.586","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.604","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.620","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.634","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.649","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.662","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.676","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.689","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.490095,"monotonic":2345737.453005293,"hcas":{"mlx5_0":{"port_xmit_data":146690530170938,"port_rcv_data":148621753894005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272613888,"port_rcv_data":2411499413670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393966486,"port_rcv_data":147222027736913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216229002,"port_rcv_data":2286119964231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.612","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.629","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.645","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.660","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.674","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.688","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.701","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.714","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.4908557,"monotonic":2345738.453766081,"hcas":{"mlx5_0":{"port_xmit_data":146690530184935,"port_rcv_data":148621768253685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272627885,"port_rcv_data":2411513773350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737393980482,"port_rcv_data":147222042096593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216243014,"port_rcv_data":2286134323911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.591","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.609","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.626","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.641","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.656","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":78.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.670","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.683","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.696","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.4916337,"monotonic":2345739.45454388,"hcas":{"mlx5_0":{"port_xmit_data":146690530184935,"port_rcv_data":148621768253685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272627885,"port_rcv_data":2411513773350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737394314802,"port_rcv_data":147222059779793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216583735,"port_rcv_data":2286152007111,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.640","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.657","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.674","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.690","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.705","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.719","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.732","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.746","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.4923606,"monotonic":2345740.455271063,"hcas":{"mlx5_0":{"port_xmit_data":146690530462245,"port_rcv_data":148621873899541,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685272898716,"port_rcv_data":2411619447040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737394725676,"port_rcv_data":147222174245984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672216999120,"port_rcv_data":2286266543450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.592","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.610","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.628","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.643","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.658","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.672","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:34.686","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:34.699","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.493139,"monotonic":2345741.456049331,"hcas":{"mlx5_0":{"port_xmit_data":146690530931337,"port_rcv_data":148621900424557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685273368785,"port_rcv_data":2411645971909,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737394879250,"port_rcv_data":147222182806991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217144603,"port_rcv_data":2286275112927,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.596","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.614","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.630","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.645","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.660","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.674","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.687","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.700","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.4939167,"monotonic":2345742.456827025,"hcas":{"mlx5_0":{"port_xmit_data":146690531095870,"port_rcv_data":148621911657857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685273535999,"port_rcv_data":2411657259325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737395334469,"port_rcv_data":147222208490081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217586275,"port_rcv_data":2286300821591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.573","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.590","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.607","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.622","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":79.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.637","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.651","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:36.664","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:36.677","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.4947166,"monotonic":2345743.457627135,"hcas":{"mlx5_0":{"port_xmit_data":146690531518911,"port_rcv_data":148621934669817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685273962063,"port_rcv_data":2411680249376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737395334562,"port_rcv_data":147222208490288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217586415,"port_rcv_data":2286300822286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.598","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":80.86,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.616","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.634","gpu_util_pct":50.0,"memory_used_mib":79917.0,"power_w":79.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.650","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":80.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.665","gpu_util_pct":0.0,"memory_used_mib":79889.0,"power_w":79.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.680","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.694","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.78,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.708","gpu_util_pct":0.0,"memory_used_mib":80019.0,"power_w":80.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.4954813,"monotonic":2345744.458391917,"hcas":{"mlx5_0":{"port_xmit_data":146690538097786,"port_rcv_data":148622219917004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685273962063,"port_rcv_data":2411680249376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737681412993,"port_rcv_data":147222215093240,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217586415,"port_rcv_data":2286300822286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.599","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":105.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.617","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":103.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.634","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.650","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":105.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.665","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":104.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.680","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":120.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.694","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.708","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":102.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.4962509,"monotonic":2345745.45916138,"hcas":{"mlx5_0":{"port_xmit_data":146690539631623,"port_rcv_data":148622288257689,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685273962063,"port_rcv_data":2411680249376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737748915476,"port_rcv_data":147222216599685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217586415,"port_rcv_data":2286300822286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.641","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":87.96,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.659","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":82.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.676","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":88.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.691","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":84.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.706","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":86.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.720","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.733","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.746","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":84.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.4969826,"monotonic":2345746.459892826,"hcas":{"mlx5_0":{"port_xmit_data":146690539641062,"port_rcv_data":148622297830809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685273971533,"port_rcv_data":2411689822496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737748924885,"port_rcv_data":147222226172805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217595808,"port_rcv_data":2286310395406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.621","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":81.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.640","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.657","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.673","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.689","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.704","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:40.718","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.732","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.497726,"monotonic":2345747.4606363,"hcas":{"mlx5_0":{"port_xmit_data":146690539655012,"port_rcv_data":148622312190489,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685273985530,"port_rcv_data":2411704182176,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737748938897,"port_rcv_data":147222240532485,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217609820,"port_rcv_data":2286324755086,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.624","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":81.25,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.643","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.660","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.675","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.690","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:41.705","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.718","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.732","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.4984684,"monotonic":2345748.461378741,"hcas":{"mlx5_0":{"port_xmit_data":146690539678774,"port_rcv_data":148622336123289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685274009291,"port_rcv_data":2411728114976,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737748964673,"port_rcv_data":147222266887655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217637813,"port_rcv_data":2286353474446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.579","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.597","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.614","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":81.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.630","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:42.645","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.660","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.674","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.687","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.4992578,"monotonic":2345749.462168322,"hcas":{"mlx5_0":{"port_xmit_data":146690539758180,"port_rcv_data":148622417494809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685274088760,"port_rcv_data":2411809486496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737749041646,"port_rcv_data":147222345836805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217712647,"port_rcv_data":2286430059406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.618","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.636","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.654","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.669","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.685","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.699","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.713","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.727","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.5000076,"monotonic":2345750.462918063,"hcas":{"mlx5_0":{"port_xmit_data":146690539809532,"port_rcv_data":148622470146969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685274140142,"port_rcv_data":2411862138656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737749093035,"port_rcv_data":147222398488965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217763998,"port_rcv_data":2286482711566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.625","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":81.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.643","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.661","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.677","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.692","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.707","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.721","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.734","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.5007513,"monotonic":2345751.46366181,"hcas":{"mlx5_0":{"port_xmit_data":146690539875298,"port_rcv_data":148622537158809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685274206048,"port_rcv_data":2411929150496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737749158724,"port_rcv_data":147222465500805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672217829687,"port_rcv_data":2286549723406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.604","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.622","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.640","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":80.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.656","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:45.671","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:45.686","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:45.700","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.713","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.5015168,"monotonic":2345752.464427321,"hcas":{"mlx5_0":{"port_xmit_data":146690540113216,"port_rcv_data":148622603439129,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685274443059,"port_rcv_data":2411995430816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737749403500,"port_rcv_data":147222531781125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672218071001,"port_rcv_data":2286616003726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.650","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.668","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.686","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.704","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.721","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.736","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.751","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.766","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.5022295,"monotonic":2345753.46513991,"hcas":{"mlx5_0":{"port_xmit_data":146690540513550,"port_rcv_data":148622698092275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685274839378,"port_rcv_data":2412090132088,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737749834524,"port_rcv_data":147222626433779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672218498072,"port_rcv_data":2286710706170,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.604","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.622","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.640","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.656","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.671","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.685","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.699","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.713","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.502997,"monotonic":2345754.46590742,"hcas":{"mlx5_0":{"port_xmit_data":146690540765611,"port_rcv_data":148622778782674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685275085813,"port_rcv_data":2412170809833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737750435691,"port_rcv_data":147222724794436,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672219096226,"port_rcv_data":2286809079545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.641","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.659","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.676","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.692","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":79.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:48.707","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.722","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.736","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.750","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.5037293,"monotonic":2345755.466639824,"hcas":{"mlx5_0":{"port_xmit_data":146690541169758,"port_rcv_data":148622863515395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685275497617,"port_rcv_data":2412255555048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737750844940,"port_rcv_data":147222809539651,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672219514940,"port_rcv_data":2286893812330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.629","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.647","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.665","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.681","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.696","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.711","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.725","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.739","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.5044768,"monotonic":2345756.46738707,"hcas":{"mlx5_0":{"port_xmit_data":146690541578028,"port_rcv_data":148622968329304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685275905245,"port_rcv_data":2412360434755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737751107246,"port_rcv_data":147222905511992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672219777231,"port_rcv_data":2286989849957,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.586","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.604","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.621","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.636","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.651","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.665","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.679","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.693","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.505267,"monotonic":2345757.468177284,"hcas":{"mlx5_0":{"port_xmit_data":146690541658535,"port_rcv_data":148623048201421,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685275985581,"port_rcv_data":2412440247932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737751689457,"port_rcv_data":147223011908289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672220351862,"port_rcv_data":2287096188978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.652","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.671","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.691","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:51.708","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:51.724","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.739","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.754","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.768","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.5059817,"monotonic":2345758.468892046,"hcas":{"mlx5_0":{"port_xmit_data":146690542297337,"port_rcv_data":148623156121802,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685276634717,"port_rcv_data":2412548373153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737751764973,"port_rcv_data":147223086934778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672220427905,"port_rcv_data":2287171274939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.625","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.644","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.661","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.86,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.677","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.693","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.707","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.721","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.735","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.5067348,"monotonic":2345759.469645181,"hcas":{"mlx5_0":{"port_xmit_data":146690542471278,"port_rcv_data":148623240309911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685276805325,"port_rcv_data":2412632537674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737752166113,"port_rcv_data":147223188840099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672220829386,"port_rcv_data":2287273389632,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.629","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.9,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.648","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.665","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.64,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.680","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.695","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":78.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.710","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.724","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.738","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.5074878,"monotonic":2345760.470398087,"hcas":{"mlx5_0":{"port_xmit_data":146690542895776,"port_rcv_data":148623320800155,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685277229700,"port_rcv_data":2412713070172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737752570669,"port_rcv_data":147223266871591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672221236602,"port_rcv_data":2287351279650,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.633","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.651","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.669","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.684","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.699","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.714","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.728","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.742","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.508234,"monotonic":2345761.471144257,"hcas":{"mlx5_0":{"port_xmit_data":146690543323297,"port_rcv_data":148623416193369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685277657643,"port_rcv_data":2412808536394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737752993260,"port_rcv_data":147223362264837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672221657396,"port_rcv_data":2287446745904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.633","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.652","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.669","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.685","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:55.700","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.715","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.729","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.743","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.5089808,"monotonic":2345762.471890857,"hcas":{"mlx5_0":{"port_xmit_data":146690543704768,"port_rcv_data":148623511176702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685278048414,"port_rcv_data":2412903566041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737753412659,"port_rcv_data":147223457523462,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672222075803,"port_rcv_data":2287541972795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.625","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.642","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.659","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.675","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.689","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.703","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.716","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.729","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":78.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.5097432,"monotonic":2345763.472653619,"hcas":{"mlx5_0":{"port_xmit_data":146690544283353,"port_rcv_data":148623607037156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685278619139,"port_rcv_data":2412999379225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737753636061,"port_rcv_data":147223535630536,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672222301576,"port_rcv_data":2287620114579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.629","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.647","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.665","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.681","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.696","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":79.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.711","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.725","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.739","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":79.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.510494,"monotonic":2345764.473404186,"hcas":{"mlx5_0":{"port_xmit_data":146690544698273,"port_rcv_data":148623704597765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685279036539,"port_rcv_data":2413096981252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737754040719,"port_rcv_data":147223632776587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672222704762,"port_rcv_data":2287717302260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.653","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":80.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.671","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.688","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.703","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":80.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:58.719","gpu_util_pct":0.0,"memory_used_mib":79891.0,"power_w":80.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:58.733","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:58.747","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:58.761","gpu_util_pct":0.0,"memory_used_mib":80021.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.5108507,"monotonic":2345765.473760441,"hcas":{"mlx5_0":{"port_xmit_data":146690546151865,"port_rcv_data":148623830073876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685279259989,"port_rcv_data":2413173703834,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147737804021307,"port_rcv_data":147223722924318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672223131400,"port_rcv_data":2287806131124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.632","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":85.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.650","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":82.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.667","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":84.81,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.683","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":86.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.698","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":85.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.712","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":84.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.726","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":84.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.740","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":85.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.5116022,"monotonic":2345766.474512455,"hcas":{"mlx5_0":{"port_xmit_data":146690553422776,"port_rcv_data":148624224020628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685279511973,"port_rcv_data":2413258998808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147738113403266,"port_rcv_data":147223829721387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672223685866,"port_rcv_data":2287906052186,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.615","gpu_util_pct":99.0,"memory_used_mib":79973.0,"power_w":122.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.633","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":122.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.650","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.666","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":122.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.681","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":122.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.695","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":121.9,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.709","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.722","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":118.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.5123703,"monotonic":2345767.475280756,"hcas":{"mlx5_0":{"port_xmit_data":146690561098832,"port_rcv_data":148624639438276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685280157129,"port_rcv_data":2413367336170,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147738420196639,"port_rcv_data":147223927410585,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672223949552,"port_rcv_data":2287996708072,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.614","gpu_util_pct":99.0,"memory_used_mib":79973.0,"power_w":124.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.632","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":122.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.650","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.665","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":120.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.680","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":121.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.695","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":119.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:01.708","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":119.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.722","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":121.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.5131402,"monotonic":2345768.476050568,"hcas":{"mlx5_0":{"port_xmit_data":146690568347076,"port_rcv_data":148625017042006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685280404912,"port_rcv_data":2413438618450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147738727731120,"port_rcv_data":147224023316770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672224519378,"port_rcv_data":2288085671292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.616","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":122.38,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.633","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.650","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.665","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":124.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.680","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":122.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.694","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":123.11,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.708","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.721","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":119.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.5139117,"monotonic":2345769.476822168,"hcas":{"mlx5_0":{"port_xmit_data":146690575959141,"port_rcv_data":148625419068955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685280986457,"port_rcv_data":2413532605051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147739035745027,"port_rcv_data":147224104083697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672224709687,"port_rcv_data":2288159581057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.614","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":124.52,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.632","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":121.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.649","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":126.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.665","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":121.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.680","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":120.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.694","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":122.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.708","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":122.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.721","gpu_util_pct":99.0,"memory_used_mib":80021.0,"power_w":121.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.5146842,"monotonic":2345770.477594566,"hcas":{"mlx5_0":{"port_xmit_data":146690583374121,"port_rcv_data":148625848438545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685281417558,"port_rcv_data":2413655906817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147739341487713,"port_rcv_data":147224236905260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672225193210,"port_rcv_data":2288285276851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.616","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":127.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.633","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.650","gpu_util_pct":87.0,"memory_used_mib":79919.0,"power_w":121.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.666","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":124.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.681","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":122.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.696","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":122.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.710","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.723","gpu_util_pct":91.0,"memory_used_mib":80021.0,"power_w":120.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.515451,"monotonic":2345771.478361436,"hcas":{"mlx5_0":{"port_xmit_data":146690594711257,"port_rcv_data":148626320712495,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685281576712,"port_rcv_data":2413665905532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147739805379548,"port_rcv_data":147224275788744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672225669912,"port_rcv_data":2288312958766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.614","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":128.57,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.632","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.650","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":128.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.665","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":125.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.680","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":123.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:05.694","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":125.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.709","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":127.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.722","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":121.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.5162227,"monotonic":2345772.47913329,"hcas":{"mlx5_0":{"port_xmit_data":146690606878837,"port_rcv_data":148626836516714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685282057972,"port_rcv_data":2413692430332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147740293679964,"port_rcv_data":147224296307915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672225828353,"port_rcv_data":2288321800366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.635","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":129.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.653","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":125.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.670","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":131.61,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.685","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":124.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.700","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":122.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.715","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":123.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.729","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":127.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.743","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":121.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.516973,"monotonic":2345773.47988353,"hcas":{"mlx5_0":{"port_xmit_data":146690618733541,"port_rcv_data":148627333791740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685282215731,"port_rcv_data":2413701271932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147740782657059,"port_rcv_data":147224334543714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672226298608,"port_rcv_data":2288348325166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.619","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":128.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.637","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.654","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":130.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.675","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":127.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.693","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":125.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.709","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":126.32,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.725","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.740","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":123.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.517734,"monotonic":2345774.480644044,"hcas":{"mlx5_0":{"port_xmit_data":146690630952472,"port_rcv_data":148627851723923,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685282751876,"port_rcv_data":2413731874508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147741271229108,"port_rcv_data":147224350476268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672226358701,"port_rcv_data":2288352404242,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.642","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":128.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.660","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.677","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":130.45,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.692","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":124.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:08.708","gpu_util_pct":100.0,"memory_used_mib":79891.0,"power_w":121.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.722","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":125.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.736","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.750","gpu_util_pct":100.0,"memory_used_mib":80021.0,"power_w":121.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.5184886,"monotonic":2345775.481397556,"hcas":{"mlx5_0":{"port_xmit_data":146690642925913,"port_rcv_data":148628439238494,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685282751876,"port_rcv_data":2413731874508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147741858193513,"port_rcv_data":147224397795604,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672226979120,"port_rcv_data":2288387770642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.604","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":130.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.622","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":127.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.639","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":130.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.655","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":127.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.670","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":126.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.685","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":127.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.698","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":127.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.712","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":125.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.5192776,"monotonic":2345776.482188156,"hcas":{"mlx5_0":{"port_xmit_data":146690655839135,"port_rcv_data":148629122132973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685283374511,"port_rcv_data":2413767240908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147742505543411,"port_rcv_data":147224410086058,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672226979120,"port_rcv_data":2288387770642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.620","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":129.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.638","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":128.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.655","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":129.77,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.671","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":128.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.686","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":127.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.700","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":128.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.713","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":131.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.727","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":124.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.520051,"monotonic":2345777.482961363,"hcas":{"mlx5_0":{"port_xmit_data":146690668015967,"port_rcv_data":148629766387900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685283374635,"port_rcv_data":2413767241184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147743152238851,"port_rcv_data":147224457146751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672227578923,"port_rcv_data":2288422584560,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.624","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":132.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.642","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":131.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.659","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":133.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.674","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":129.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.689","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":126.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.704","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":131.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:11.718","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":128.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.731","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":126.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.5208244,"monotonic":2345778.483734669,"hcas":{"mlx5_0":{"port_xmit_data":146690680865035,"port_rcv_data":148630443984891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685283968626,"port_rcv_data":2413802054452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147743793221138,"port_rcv_data":147224469350315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672227579109,"port_rcv_data":2288422585486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.623","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":129.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.641","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":129.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.658","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":132.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.674","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":132.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:12.688","gpu_util_pct":100.0,"memory_used_mib":79893.0,"power_w":125.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:12.703","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":129.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.717","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":128.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.730","gpu_util_pct":100.0,"memory_used_mib":80023.0,"power_w":126.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.5215929,"monotonic":2345779.484503258,"hcas":{"mlx5_0":{"port_xmit_data":146690692737690,"port_rcv_data":148631180632738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685284133081,"port_rcv_data":2413810896052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147744521557017,"port_rcv_data":147224507621864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672228050309,"port_rcv_data":2288449110286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.621","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":136.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.635","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":130.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.647","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.659","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.669","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":126.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.680","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":129.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.689","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.699","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":128.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.5224068,"monotonic":2345780.485317348,"hcas":{"mlx5_0":{"port_xmit_data":146690704597981,"port_rcv_data":148631976888004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685284601305,"port_rcv_data":2413837420852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147745291533915,"port_rcv_data":147224527889377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672228204534,"port_rcv_data":2288457951886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.603","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":135.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.616","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.629","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.641","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.652","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":126.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.663","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":130.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.673","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.683","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":132.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.523242,"monotonic":2345781.486152469,"hcas":{"mlx5_0":{"port_xmit_data":146690716410490,"port_rcv_data":148632775490564,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685284757049,"port_rcv_data":2413846262452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147746083421264,"port_rcv_data":147224566154241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672228681547,"port_rcv_data":2288484476686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.621","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":133.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.634","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.647","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.659","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.671","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":131.4,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:15.682","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:15.692","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":133.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.702","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":129.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.5240521,"monotonic":2345782.486962724,"hcas":{"mlx5_0":{"port_xmit_data":146690728298320,"port_rcv_data":148633571487027,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685285231907,"port_rcv_data":2413872787252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147746851672287,"port_rcv_data":147224586408025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672228837539,"port_rcv_data":2288493318286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.626","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":130.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.639","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":132.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.652","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":135.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.664","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.675","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":130.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.685","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.695","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.707","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":128.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.5248585,"monotonic":2345783.487768626,"hcas":{"mlx5_0":{"port_xmit_data":146690740156506,"port_rcv_data":148634370194594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685285543162,"port_rcv_data":2413891575372,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147747634018085,"port_rcv_data":147224616800725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672229151708,"port_rcv_data":2288512107706,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.641","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":134.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.654","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":130.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.667","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":137.58,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.679","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.690","gpu_util_pct":100.0,"memory_used_mib":79895.0,"power_w":127.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.700","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":133.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.710","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.720","gpu_util_pct":100.0,"memory_used_mib":80025.0,"power_w":132.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.5256505,"monotonic":2345784.488560894,"hcas":{"mlx5_0":{"port_xmit_data":146690750785704,"port_rcv_data":148635159278817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685285543286,"port_rcv_data":2413891575648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147748421840323,"port_rcv_data":147224663355041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672229768252,"port_rcv_data":2288548026808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.643","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.656","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.669","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.681","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.692","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.15,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.703","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.713","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.723","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.526442,"monotonic":2345785.489352313,"hcas":{"mlx5_0":{"port_xmit_data":146690761788376,"port_rcv_data":148636019503693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286153862,"port_rcv_data":2413927494100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147749246876546,"port_rcv_data":147224673783657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672229768438,"port_rcv_data":2288548027734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.625","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.639","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.651","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.663","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.674","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.684","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.694","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.704","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.5272546,"monotonic":2345786.490164839,"hcas":{"mlx5_0":{"port_xmit_data":146690771795525,"port_rcv_data":148636874594992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286153986,"port_rcv_data":2413927494376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147750103656373,"port_rcv_data":147224719194476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230369094,"port_rcv_data":2288583394244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.608","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.621","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.634","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.84,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.646","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.656","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.72,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:20.667","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.677","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.687","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.52809,"monotonic":2345787.491000499,"hcas":{"mlx5_0":{"port_xmit_data":146690782370021,"port_rcv_data":148637823264853,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286756843,"port_rcv_data":2413962860305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147751017291237,"port_rcv_data":147224729671773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230374860,"port_rcv_data":2288584102581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.628","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.641","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.654","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.666","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.677","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.56,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.693","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.705","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.716","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.5288877,"monotonic":2345788.491798149,"hcas":{"mlx5_0":{"port_xmit_data":146690791855151,"port_rcv_data":148638729482402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286767275,"port_rcv_data":2413964274684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147751920217153,"port_rcv_data":147224740034053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230380409,"port_rcv_data":2288584810918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.608","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.621","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.634","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.645","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.656","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:22.666","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.676","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.685","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.5297227,"monotonic":2345789.492633054,"hcas":{"mlx5_0":{"port_xmit_data":146690799973738,"port_rcv_data":148639639273876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286767275,"port_rcv_data":2413964274684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147752832253213,"port_rcv_data":147224748217939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230380409,"port_rcv_data":2288584810918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.645","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":128.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.658","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.671","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.682","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.693","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.704","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.714","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.723","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":124.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.5305147,"monotonic":2345790.493425145,"hcas":{"mlx5_0":{"port_xmit_data":146690807700416,"port_rcv_data":148640501333486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286767275,"port_rcv_data":2413964274684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147753692568245,"port_rcv_data":147224755956193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230380409,"port_rcv_data":2288584810918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.645","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.659","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.671","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.684","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:24.695","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.705","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.715","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.725","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":124.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.531307,"monotonic":2345791.494217381,"hcas":{"mlx5_0":{"port_xmit_data":146690815995055,"port_rcv_data":148641429874463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286767275,"port_rcv_data":2413964274684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147754623338360,"port_rcv_data":147224764307846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230380409,"port_rcv_data":2288584810918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.629","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.642","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.654","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.666","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.676","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.687","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.697","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.706","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":126.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.5321174,"monotonic":2345792.49502774,"hcas":{"mlx5_0":{"port_xmit_data":146690823673830,"port_rcv_data":148642286678800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286767275,"port_rcv_data":2413964274684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147755479591308,"port_rcv_data":147224772020585,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230380409,"port_rcv_data":2288584810918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.629","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.642","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.655","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.667","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:26.679","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.690","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.700","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.709","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":125.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.5329247,"monotonic":2345793.495835348,"hcas":{"mlx5_0":{"port_xmit_data":146690831948873,"port_rcv_data":148643215272889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286767275,"port_rcv_data":2413964274684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147756408436560,"port_rcv_data":147224780340100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230380409,"port_rcv_data":2288584810918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.613","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.96,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.626","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.640","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.652","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.663","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.674","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.684","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.693","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.5337493,"monotonic":2345794.496659648,"hcas":{"mlx5_0":{"port_xmit_data":146690840743641,"port_rcv_data":148644147921884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286776683,"port_rcv_data":2413973847804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147757330602186,"port_rcv_data":147224798724849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230389895,"port_rcv_data":2288594384038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.649","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.662","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:28.674","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:28.686","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:28.697","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:28.707","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.717","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.727","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.5345447,"monotonic":2345795.49745507,"hcas":{"mlx5_0":{"port_xmit_data":146690849362560,"port_rcv_data":148645039345036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286790587,"port_rcv_data":2413988207484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147758208798441,"port_rcv_data":147224821727087,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230403876,"port_rcv_data":2288608743718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.614","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.627","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.640","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.26,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.652","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.662","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.672","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.682","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.692","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.5353734,"monotonic":2345796.498284131,"hcas":{"mlx5_0":{"port_xmit_data":146690858755121,"port_rcv_data":148646018236048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286804645,"port_rcv_data":2414002567164,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147759171210860,"port_rcv_data":147224845484881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230417950,"port_rcv_data":2288623103398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.632","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.645","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.657","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.669","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:30.680","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:30.690","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.701","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.710","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.5361843,"monotonic":2345797.499094804,"hcas":{"mlx5_0":{"port_xmit_data":146690867692136,"port_rcv_data":148646945669567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286818580,"port_rcv_data":2414016926844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147760086880850,"port_rcv_data":147224868834727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230431962,"port_rcv_data":2288637463078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.633","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.647","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.659","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.671","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.681","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.692","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.702","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.711","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.5369954,"monotonic":2345798.499906011,"hcas":{"mlx5_0":{"port_xmit_data":146690876944334,"port_rcv_data":148647920033883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286846743,"port_rcv_data":2414045646204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147761031715374,"port_rcv_data":147224906810265,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230459939,"port_rcv_data":2288666182438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.616","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.629","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.643","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:32.655","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:32.666","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.676","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.686","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.696","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.5378306,"monotonic":2345799.500741025,"hcas":{"mlx5_0":{"port_xmit_data":146690886145735,"port_rcv_data":148648876420443,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286860755,"port_rcv_data":2414060005884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147761972204562,"port_rcv_data":147224930368522,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230473951,"port_rcv_data":2288680542118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.618","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.631","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.644","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.656","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.667","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.52,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.678","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.688","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.697","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.5386596,"monotonic":2345800.501570056,"hcas":{"mlx5_0":{"port_xmit_data":146690895349317,"port_rcv_data":148649819680375,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685286860755,"port_rcv_data":2414060005884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147762915716565,"port_rcv_data":147224948455905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672230652651,"port_rcv_data":2288689383718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.618","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.631","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.644","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.656","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:34.667","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.678","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.688","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.697","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.5394895,"monotonic":2345801.502399978,"hcas":{"mlx5_0":{"port_xmit_data":146690904377536,"port_rcv_data":148650788005923,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685287270389,"port_rcv_data":2414148658870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147763795904489,"port_rcv_data":147225045710290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231065044,"port_rcv_data":2288778038088,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.619","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.634","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.647","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.13,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.659","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.670","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.27,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.681","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.74,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.691","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.701","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.5403166,"monotonic":2345802.503227233,"hcas":{"mlx5_0":{"port_xmit_data":146690914335063,"port_rcv_data":148651825819294,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685287614396,"port_rcv_data":2414196702637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147764787827158,"port_rcv_data":147225103575438,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231402448,"port_rcv_data":2288826219961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.641","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.654","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.667","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.678","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.689","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.699","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.709","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.718","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.5411263,"monotonic":2345803.504036777,"hcas":{"mlx5_0":{"port_xmit_data":146690923453412,"port_rcv_data":148652743260173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685287920552,"port_rcv_data":2414214662001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147765685942182,"port_rcv_data":147225130353570,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231704496,"port_rcv_data":2288844179975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.637","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.650","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.663","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.21,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.675","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.686","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.25,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.697","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.707","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.717","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.5419388,"monotonic":2345804.504849094,"hcas":{"mlx5_0":{"port_xmit_data":146690933193494,"port_rcv_data":148653731592809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147766657041101,"port_rcv_data":147225148820669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.642","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.655","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.668","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:38.680","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:38.691","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.702","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.712","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.721","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.5427551,"monotonic":2345805.505665627,"hcas":{"mlx5_0":{"port_xmit_data":146690941300611,"port_rcv_data":148654591982431,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147767519121446,"port_rcv_data":147225156981948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.639","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.652","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.665","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.676","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.687","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.698","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.19,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.708","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.718","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.5435762,"monotonic":2345806.506486643,"hcas":{"mlx5_0":{"port_xmit_data":146690948810322,"port_rcv_data":148655426678503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147768351734886,"port_rcv_data":147225164493062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.640","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.07,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.654","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.666","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.678","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.690","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.700","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.711","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.720","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":126.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.544391,"monotonic":2345807.507301266,"hcas":{"mlx5_0":{"port_xmit_data":146690957411765,"port_rcv_data":148656394177459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147769319302512,"port_rcv_data":147225173139626,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.644","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.657","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:41.670","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:41.682","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.693","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":123.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.703","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.714","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.723","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.5452003,"monotonic":2345808.508109905,"hcas":{"mlx5_0":{"port_xmit_data":146690965283519,"port_rcv_data":148657273116664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147770200851177,"port_rcv_data":147225181071343,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.662","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":130.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.675","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.688","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.700","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.711","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.722","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.732","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.741","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.5459907,"monotonic":2345809.508901205,"hcas":{"mlx5_0":{"port_xmit_data":146690973483882,"port_rcv_data":148658191956972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147771117742240,"port_rcv_data":147225189290272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.625","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.638","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.651","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:43.663","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:43.674","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.684","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.694","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.703","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.5468225,"monotonic":2345810.509733262,"hcas":{"mlx5_0":{"port_xmit_data":146690981564878,"port_rcv_data":148659097599445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147772024815078,"port_rcv_data":147225197425395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.620","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.633","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.646","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.33,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.656","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.667","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":123.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.677","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.686","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.696","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.547667,"monotonic":2345811.510577369,"hcas":{"mlx5_0":{"port_xmit_data":146690989267030,"port_rcv_data":148659958976471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147772886481942,"port_rcv_data":147225205157718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.628","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.38,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.641","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.654","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.666","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.678","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":124.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.688","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.698","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.708","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.548495,"monotonic":2345812.511405443,"hcas":{"mlx5_0":{"port_xmit_data":146690997428524,"port_rcv_data":148660872747865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147773800259798,"port_rcv_data":147225213343451,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.647","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.660","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.673","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.685","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.696","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.707","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.719","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.730","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":124.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.5493004,"monotonic":2345813.512211048,"hcas":{"mlx5_0":{"port_xmit_data":146691005506539,"port_rcv_data":148661778209809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147774705767006,"port_rcv_data":147225221458822,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.666","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.679","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.692","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.703","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.713","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.724","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.733","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.743","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.5500956,"monotonic":2345814.513006012,"hcas":{"mlx5_0":{"port_xmit_data":146691013071303,"port_rcv_data":148662620478818,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288223515,"port_rcv_data":2414232621296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147775548361904,"port_rcv_data":147225229043681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231856164,"port_rcv_data":2288853160214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.648","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.661","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.674","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.41,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:48.686","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:48.697","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":123.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.707","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.717","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.727","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.5509014,"monotonic":2345815.513812346,"hcas":{"mlx5_0":{"port_xmit_data":146691022371430,"port_rcv_data":148663604012174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288233187,"port_rcv_data":2414242194416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147776521016629,"port_rcv_data":147225247935044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231865572,"port_rcv_data":2288862733334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.665","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.678","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:49.691","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.21,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.703","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.714","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.7,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.725","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.735","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.744","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.5509834,"monotonic":2345816.513893807,"hcas":{"mlx5_0":{"port_xmit_data":146691031357595,"port_rcv_data":148664538877209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288247199,"port_rcv_data":2414256554096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147777442818471,"port_rcv_data":147225271318488,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231879584,"port_rcv_data":2288877093014,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.631","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.645","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.659","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.671","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.682","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.692","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.702","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.712","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.5518157,"monotonic":2345817.514726006,"hcas":{"mlx5_0":{"port_xmit_data":146691040579925,"port_rcv_data":148665498033153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288261227,"port_rcv_data":2414270913776,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147778387696447,"port_rcv_data":147225294921508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231893596,"port_rcv_data":2288891452694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.665","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.679","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.697","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.710","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.723","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.735","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.746","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.757","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.5526018,"monotonic":2345818.515512281,"hcas":{"mlx5_0":{"port_xmit_data":146691050099859,"port_rcv_data":148666502654702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288289297,"port_rcv_data":2414299633136,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147779363286497,"port_rcv_data":147225333175053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231921620,"port_rcv_data":2288920172054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.652","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.666","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.679","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.690","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.702","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.712","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.61,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.722","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.732","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.553112,"monotonic":2345819.516022565,"hcas":{"mlx5_0":{"port_xmit_data":146691059410357,"port_rcv_data":148667472241587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288303232,"port_rcv_data":2414313992816,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147780318971345,"port_rcv_data":147225356881756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231935555,"port_rcv_data":2288934531734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.652","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.665","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.677","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.689","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.700","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.710","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.720","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.729","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.5539289,"monotonic":2345820.516839398,"hcas":{"mlx5_0":{"port_xmit_data":146691068766552,"port_rcv_data":148668444660202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288317399,"port_rcv_data":2414328352496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147781274935642,"port_rcv_data":147225380594557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672231949675,"port_rcv_data":2288948891414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.670","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.683","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:54.696","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.708","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.720","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.730","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.741","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.750","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.5547197,"monotonic":2345821.51763036,"hcas":{"mlx5_0":{"port_xmit_data":146691077990961,"port_rcv_data":148669393063060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288317399,"port_rcv_data":2414328352496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147782225777381,"port_rcv_data":147225398725993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672232123415,"port_rcv_data":2288957733014,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.652","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.665","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.678","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.690","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.701","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.711","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.721","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.731","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.41,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.5555327,"monotonic":2345822.518443167,"hcas":{"mlx5_0":{"port_xmit_data":146691087582219,"port_rcv_data":148670459896102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685288585766,"port_rcv_data":2414438524380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147783180791147,"port_rcv_data":147225527017121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672232553571,"port_rcv_data":2289076747882,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.635","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.648","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.661","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.673","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.684","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.694","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.704","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.714","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.556363,"monotonic":2345823.519273455,"hcas":{"mlx5_0":{"port_xmit_data":146691097563390,"port_rcv_data":148671465402230,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289055989,"port_rcv_data":2414465049180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147784161832014,"port_rcv_data":147225545444020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672232709811,"port_rcv_data":2289085589482,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.635","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.649","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.661","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.673","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.684","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.695","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.704","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.714","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.5571098,"monotonic":2345824.520020086,"hcas":{"mlx5_0":{"port_xmit_data":146691106497369,"port_rcv_data":148672380485861,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289056082,"port_rcv_data":2414465049387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147785077729106,"port_rcv_data":147225581360994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233168735,"port_rcv_data":2289112528808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.656","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.669","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.682","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.695","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.706","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.717","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.727","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.737","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.5579195,"monotonic":2345825.520830017,"hcas":{"mlx5_0":{"port_xmit_data":146691115742986,"port_rcv_data":148673312629443,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289361293,"port_rcv_data":2414483008682,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147785992040632,"port_rcv_data":147225599306498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233323254,"port_rcv_data":2289121509047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.673","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.686","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.699","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.710","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.722","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.732","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.742","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.752","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.5587149,"monotonic":2345826.521625427,"hcas":{"mlx5_0":{"port_xmit_data":146691124993584,"port_rcv_data":148674246453252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289666565,"port_rcv_data":2414500967908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147786907713726,"port_rcv_data":147225608280360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233323347,"port_rcv_data":2289121509510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.638","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.651","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.664","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.676","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.687","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.698","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.708","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.717","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.5595438,"monotonic":2345827.522454277,"hcas":{"mlx5_0":{"port_xmit_data":146691133191817,"port_rcv_data":148675151949108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289666565,"port_rcv_data":2414500967908,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147787811767310,"port_rcv_data":147225616490787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233323347,"port_rcv_data":2289121509510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.656","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.669","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.683","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:01.695","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.706","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.717","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.727","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.737","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.5603552,"monotonic":2345828.523265667,"hcas":{"mlx5_0":{"port_xmit_data":146691142190175,"port_rcv_data":148676091756828,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289676005,"port_rcv_data":2414510541028,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147788743947974,"port_rcv_data":147225635113803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233332787,"port_rcv_data":2289131082630,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.660","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.672","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.685","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.697","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.707","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.718","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.727","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.737","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.5611653,"monotonic":2345829.524075984,"hcas":{"mlx5_0":{"port_xmit_data":146691151253788,"port_rcv_data":148677033628683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289689986,"port_rcv_data":2414524900708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147789671487516,"port_rcv_data":147225658561508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233346799,"port_rcv_data":2289145442310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.641","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.38,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.654","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.667","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:03.679","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.691","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.702","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.712","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.722","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.561993,"monotonic":2345830.524903481,"hcas":{"mlx5_0":{"port_xmit_data":146691160589356,"port_rcv_data":148678016535043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289718072,"port_rcv_data":2414553620068,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147790624118725,"port_rcv_data":147225696601910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233374993,"port_rcv_data":2289174161670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.660","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.673","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.686","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.697","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.708","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.718","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.728","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.737","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.562811,"monotonic":2345831.525721224,"hcas":{"mlx5_0":{"port_xmit_data":146691169695665,"port_rcv_data":148678965229434,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289732208,"port_rcv_data":2414567979748,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147791558432417,"port_rcv_data":147225720087865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233389067,"port_rcv_data":2289188521350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.677","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.690","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.703","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.715","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.726","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.16,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.737","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.32,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.747","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.757","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.5636098,"monotonic":2345832.526519184,"hcas":{"mlx5_0":{"port_xmit_data":146691178411757,"port_rcv_data":148679869520355,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289746173,"port_rcv_data":2414582339428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147792449955087,"port_rcv_data":147225743203284,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233403033,"port_rcv_data":2289202881030,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.660","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.674","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.687","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.699","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.710","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.720","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.731","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.740","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.564421,"monotonic":2345833.52733137,"hcas":{"mlx5_0":{"port_xmit_data":146691187789744,"port_rcv_data":148680844854907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289761038,"port_rcv_data":2414596699108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147793409173625,"port_rcv_data":147225766942292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233417091,"port_rcv_data":2289217240710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.661","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.676","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.689","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.701","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.712","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.18,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.723","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.733","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.743","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.5652282,"monotonic":2345834.52813885,"hcas":{"mlx5_0":{"port_xmit_data":146691197146129,"port_rcv_data":148681806723353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685289761038,"port_rcv_data":2414596699108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147794371458727,"port_rcv_data":147225794018629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672233764710,"port_rcv_data":2289234923910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.646","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.659","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.673","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.685","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.696","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.66,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.707","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.717","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.727","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.5660527,"monotonic":2345835.528963187,"hcas":{"mlx5_0":{"port_xmit_data":146691206784539,"port_rcv_data":148682878858458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685290025685,"port_rcv_data":2414705360108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147795337551350,"port_rcv_data":147225920940286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234203344,"port_rcv_data":2289352424782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.665","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.679","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:09.692","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:09.704","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.716","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.727","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.737","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.747","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.5668578,"monotonic":2345836.529768259,"hcas":{"mlx5_0":{"port_xmit_data":146691216650235,"port_rcv_data":148683877123601,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685290490514,"port_rcv_data":2414731884977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147796308126683,"port_rcv_data":147225939199347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234354097,"port_rcv_data":2289361266409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.682","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.695","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.708","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.720","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.732","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.742","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.752","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.762","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.56765,"monotonic":2345837.530560409,"hcas":{"mlx5_0":{"port_xmit_data":146691225939218,"port_rcv_data":148684814997109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685290644057,"port_rcv_data":2414740726649,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147797237297094,"port_rcv_data":147225974887865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234812231,"port_rcv_data":2289387791523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.664","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.677","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.691","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.703","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.714","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.725","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.735","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.744","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.5684593,"monotonic":2345838.531369765,"hcas":{"mlx5_0":{"port_xmit_data":146691235468759,"port_rcv_data":148685770486979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291098517,"port_rcv_data":2414767251044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147798168393238,"port_rcv_data":147225984011606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234812370,"port_rcv_data":2289387792218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.668","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.682","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.694","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.706","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.717","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.728","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.738","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.747","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.5692682,"monotonic":2345839.532178804,"hcas":{"mlx5_0":{"port_xmit_data":146691244798857,"port_rcv_data":148686733762762,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291098517,"port_rcv_data":2414767251044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147799131596994,"port_rcv_data":147225993392538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234812370,"port_rcv_data":2289387792218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.684","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.698","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:13.711","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":135.84,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.723","gpu_util_pct":98.0,"memory_used_mib":79963.0,"power_w":136.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.734","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.745","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.765","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.5700598,"monotonic":2345840.532970358,"hcas":{"mlx5_0":{"port_xmit_data":146691252677621,"port_rcv_data":148687607001339,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291098517,"port_rcv_data":2414767251044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147800003025806,"port_rcv_data":147226001275481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234812370,"port_rcv_data":2289387792218,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.650","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.81,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.664","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.676","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.688","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.700","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.23,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.710","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.720","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.731","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.570887,"monotonic":2345841.533797619,"hcas":{"mlx5_0":{"port_xmit_data":146691262076512,"port_rcv_data":148688583020751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291107988,"port_rcv_data":2414776824164,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147800971433624,"port_rcv_data":147226020302045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234821763,"port_rcv_data":2289397365338,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.668","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.682","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.695","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.707","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.719","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.730","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.740","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.750","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.5716937,"monotonic":2345842.534604094,"hcas":{"mlx5_0":{"port_xmit_data":146691271457810,"port_rcv_data":148689557765200,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291122000,"port_rcv_data":2414791183844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147801930139077,"port_rcv_data":147226044042947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234835806,"port_rcv_data":2289411725018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.685","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:16.699","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:16.714","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.33,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.727","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.739","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.38,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.750","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.761","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.771","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.57248,"monotonic":2345843.535390361,"hcas":{"mlx5_0":{"port_xmit_data":146691280660370,"port_rcv_data":148690529076404,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291150163,"port_rcv_data":2414819903204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147802872678972,"port_rcv_data":147226081977324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234864078,"port_rcv_data":2289440444378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.673","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.687","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.700","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.711","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.723","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.77,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.733","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.743","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.753","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.5732849,"monotonic":2345844.536195237,"hcas":{"mlx5_0":{"port_xmit_data":146691289796020,"port_rcv_data":148691478746985,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291164175,"port_rcv_data":2414834262884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147803809474865,"port_rcv_data":147226105506879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234878044,"port_rcv_data":2289454804058,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.672","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.31,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.685","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:18.698","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.711","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.722","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.733","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.744","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.754","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.5740905,"monotonic":2345845.537000844,"hcas":{"mlx5_0":{"port_xmit_data":146691298940190,"port_rcv_data":148692428422335,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291178094,"port_rcv_data":2414848622564,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147804743103234,"port_rcv_data":147226129010976,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234892056,"port_rcv_data":2289469163738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.689","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.703","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.716","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.728","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.739","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.749","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.759","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.769","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.5748827,"monotonic":2345846.537793302,"hcas":{"mlx5_0":{"port_xmit_data":146691307994183,"port_rcv_data":148693364610847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291192277,"port_rcv_data":2414862982244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147805666252333,"port_rcv_data":147226152455192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672234906068,"port_rcv_data":2289483523418,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.677","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:20.690","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:20.703","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.05,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.715","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.726","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.736","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.746","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.756","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.575687,"monotonic":2345847.538597584,"hcas":{"mlx5_0":{"port_xmit_data":146691318100606,"port_rcv_data":148694305461049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291192277,"port_rcv_data":2414862982244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147806608339570,"port_rcv_data":147226180290062,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672235226065,"port_rcv_data":2289501206618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.656","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.669","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.682","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.695","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.708","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.720","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.731","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.742","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.5765052,"monotonic":2345848.539415662,"hcas":{"mlx5_0":{"port_xmit_data":146691328512128,"port_rcv_data":148695357419627,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291534083,"port_rcv_data":2414976729616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147807544979411,"port_rcv_data":147226306373114,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672235637172,"port_rcv_data":2289617196734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.657","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.670","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.683","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.695","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.706","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.717","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.727","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.737","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.5773335,"monotonic":2345849.540243902,"hcas":{"mlx5_0":{"port_xmit_data":146691339371037,"port_rcv_data":148696307843045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685291947499,"port_rcv_data":2415020587045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147808453284585,"port_rcv_data":147226349504024,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672235803998,"port_rcv_data":2289649833013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.693","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.707","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.720","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.19,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.732","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.743","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.62,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.753","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.763","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.773","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.5781245,"monotonic":2345850.541034688,"hcas":{"mlx5_0":{"port_xmit_data":146691350390972,"port_rcv_data":148697208049060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685292108854,"port_rcv_data":2415043650180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147809330199016,"port_rcv_data":147226392136309,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672236118834,"port_rcv_data":2289681599884,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.677","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.690","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.703","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.716","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.727","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.28,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.738","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.748","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.758","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.5789304,"monotonic":2345851.541841031,"hcas":{"mlx5_0":{"port_xmit_data":146691361761462,"port_rcv_data":148698130188429,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685292574458,"port_rcv_data":2415083912793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147810211351707,"port_rcv_data":147226426114105,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672236281398,"port_rcv_data":2289704663506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.678","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.691","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.704","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.717","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.728","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.739","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.750","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.760","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.579741,"monotonic":2345852.542651548,"hcas":{"mlx5_0":{"port_xmit_data":146691372303584,"port_rcv_data":148699083194391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685292602932,"port_rcv_data":2415112839240,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147811135522143,"port_rcv_data":147226465381596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672236309701,"port_rcv_data":2289733383098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.678","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.692","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.704","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.716","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.728","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.738","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.748","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.758","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.5805473,"monotonic":2345853.54345792,"hcas":{"mlx5_0":{"port_xmit_data":146691383386112,"port_rcv_data":148700180119918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685292616990,"port_rcv_data":2415127198920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147812218142681,"port_rcv_data":147226490831500,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672236323713,"port_rcv_data":2289747742778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.679","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.692","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.706","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.717","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.728","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.739","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.749","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.759","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.5811095,"monotonic":2345854.544020126,"hcas":{"mlx5_0":{"port_xmit_data":146691394434667,"port_rcv_data":148701261076001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685292616990,"port_rcv_data":2415127198920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147813300937286,"port_rcv_data":147226510763016,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672236508194,"port_rcv_data":2289756584378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.681","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.694","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.707","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.07,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:28.719","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.730","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.741","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.751","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.761","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.5819156,"monotonic":2345855.544826256,"hcas":{"mlx5_0":{"port_xmit_data":146691405711286,"port_rcv_data":148702393067852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685292819172,"port_rcv_data":2415175460154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147814385500770,"port_rcv_data":147226578904840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672236888564,"port_rcv_data":2289813689556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.680","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.694","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.707","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.719","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.730","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.741","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.751","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.760","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.582723,"monotonic":2345856.545633191,"hcas":{"mlx5_0":{"port_xmit_data":146691417506764,"port_rcv_data":148703401119648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685293213446,"port_rcv_data":2415223331406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147815344003739,"port_rcv_data":147226623468224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672237078052,"port_rcv_data":2289846814968,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.663","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.676","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.689","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.701","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.712","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.38,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.723","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.733","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.744","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.5835483,"monotonic":2345857.546459031,"hcas":{"mlx5_0":{"port_xmit_data":146691429033142,"port_rcv_data":148704418567024,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685293388255,"port_rcv_data":2415284452128,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147816302412831,"port_rcv_data":147226693060548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672237137556,"port_rcv_data":2289904997370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.664","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.677","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.690","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.702","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.713","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.723","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.733","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.743","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.5843725,"monotonic":2345858.547282888,"hcas":{"mlx5_0":{"port_xmit_data":146691440491536,"port_rcv_data":148705413358767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685293417937,"port_rcv_data":2415313523012,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147817266123232,"port_rcv_data":147226733599181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672237167130,"port_rcv_data":2289934068254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.684","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.698","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:32.711","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.43,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:32.723","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.734","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.745","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.765","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.585177,"monotonic":2345859.548087217,"hcas":{"mlx5_0":{"port_xmit_data":146691451865567,"port_rcv_data":148706387109585,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685293432151,"port_rcv_data":2415327882761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147818226479826,"port_rcv_data":147226776560749,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672237490755,"port_rcv_data":2289965627643,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.666","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.679","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.693","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.705","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.716","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.727","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.738","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.748","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.5860004,"monotonic":2345860.548910891,"hcas":{"mlx5_0":{"port_xmit_data":146691463521803,"port_rcv_data":148707375249609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685293749513,"port_rcv_data":2415359649213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147819184981843,"port_rcv_data":147226811216308,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672237656977,"port_rcv_data":2289988898515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.685","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.698","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.711","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.723","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.735","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.68,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.746","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.757","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.767","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.5868063,"monotonic":2345861.549716696,"hcas":{"mlx5_0":{"port_xmit_data":146691475265933,"port_rcv_data":148708371425191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294083120,"port_rcv_data":2415391415596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147820148852186,"port_rcv_data":147226845709487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672237837056,"port_rcv_data":2290011962138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.702","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.716","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:35.729","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:35.741","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.753","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.764","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.774","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.784","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.5875947,"monotonic":2345862.550505264,"hcas":{"mlx5_0":{"port_xmit_data":146691486282575,"port_rcv_data":148709410448394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294083120,"port_rcv_data":2415391415596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147821187944598,"port_rcv_data":147226883276410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672238359328,"port_rcv_data":2290038486938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.702","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.716","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.729","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.741","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.752","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.84,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.762","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.772","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.782","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.588386,"monotonic":2345863.551296661,"hcas":{"mlx5_0":{"port_xmit_data":146691497872502,"port_rcv_data":148710554622193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294605268,"port_rcv_data":2415453024550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147822269403079,"port_rcv_data":147226929426439,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672238397009,"port_rcv_data":2290073572136,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.688","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.702","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:37.715","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.726","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.738","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.72,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.748","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.758","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.768","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.5891154,"monotonic":2345864.55202576,"hcas":{"mlx5_0":{"port_xmit_data":146691509028106,"port_rcv_data":148711540323822,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294790974,"port_rcv_data":2415491331450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147823218454402,"port_rcv_data":147226986264697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672238595238,"port_rcv_data":2290119467769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.670","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.684","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.697","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.709","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.721","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.731","gpu_util_pct":87.0,"memory_used_mib":79959.0,"power_w":137.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.741","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.751","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.5892382,"monotonic":2345865.552148571,"hcas":{"mlx5_0":{"port_xmit_data":146691522438342,"port_rcv_data":148712553448100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294851408,"port_rcv_data":2415550403942,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147824172107488,"port_rcv_data":147227051123973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672238647768,"port_rcv_data":2290170951868,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.689","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.702","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.715","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.727","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.738","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.09,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.749","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.759","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.769","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.589559,"monotonic":2345866.55246956,"hcas":{"mlx5_0":{"port_xmit_data":146691535755526,"port_rcv_data":148713536269379,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294881587,"port_rcv_data":2415579947572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147825126488284,"port_rcv_data":147227093984256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672238678737,"port_rcv_data":2290200495498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.689","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.703","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.716","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.729","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.740","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.751","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.762","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.772","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.5903609,"monotonic":2345867.553271311,"hcas":{"mlx5_0":{"port_xmit_data":146691549282259,"port_rcv_data":148714528506776,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294911998,"port_rcv_data":2415609483800,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147826088521736,"port_rcv_data":147227137030993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672238708900,"port_rcv_data":2290230031726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.689","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.703","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.716","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.728","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.739","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.76,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.750","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.760","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.770","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.5911658,"monotonic":2345868.554076342,"hcas":{"mlx5_0":{"port_xmit_data":146691562624019,"port_rcv_data":148715495535368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685294926816,"port_rcv_data":2415623843618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147827041866833,"port_rcv_data":147227182696760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672239067926,"port_rcv_data":2290262350957,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.706","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.5,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:42.720","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:42.734","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.746","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.757","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.768","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.779","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.788","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.591954,"monotonic":2345869.554864411,"hcas":{"mlx5_0":{"port_xmit_data":146691576272857,"port_rcv_data":148716466147286,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685295263274,"port_rcv_data":2415641802913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147827994478045,"port_rcv_data":147227205001344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672239225546,"port_rcv_data":2290271331195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.689","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.703","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.716","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.728","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.739","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.03,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.749","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.759","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.769","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.592761,"monotonic":2345870.555671731,"hcas":{"mlx5_0":{"port_xmit_data":146691589979686,"port_rcv_data":148717490880075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685295660136,"port_rcv_data":2415713512056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147828948850171,"port_rcv_data":147227289230817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672239581891,"port_rcv_data":2290342341326,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.692","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.706","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.719","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.731","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.742","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.4,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.752","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.763","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.773","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.5931096,"monotonic":2345871.55601995,"hcas":{"mlx5_0":{"port_xmit_data":146691602619491,"port_rcv_data":148718587035693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685295873664,"port_rcv_data":2415775526322,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147829983410733,"port_rcv_data":147227373128595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672239972382,"port_rcv_data":2290413755720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:45.692","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:45.706","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:45.720","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.732","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.743","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.754","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.765","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.775","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.5939116,"monotonic":2345872.556822129,"hcas":{"mlx5_0":{"port_xmit_data":146691615659232,"port_rcv_data":148719607370314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685296225948,"port_rcv_data":2415831769748,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147830947272836,"port_rcv_data":147227433226669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672240169899,"port_rcv_data":2290461158846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.693","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.706","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.722","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.96,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.735","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.748","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.759","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.770","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.780","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.5947099,"monotonic":2345873.557620323,"hcas":{"mlx5_0":{"port_xmit_data":146691628535284,"port_rcv_data":148720516896078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685296411964,"port_rcv_data":2415869330846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147831818121056,"port_rcv_data":147227491897683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672240498142,"port_rcv_data":2290507147001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.693","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.707","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.720","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.19,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.732","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.744","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.755","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.765","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.775","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.5955153,"monotonic":2345874.558425688,"hcas":{"mlx5_0":{"port_xmit_data":146691641354808,"port_rcv_data":148721422916059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685296591066,"port_rcv_data":2415906684579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147832687499625,"port_rcv_data":147227550550631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672240824820,"port_rcv_data":2290553135387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.692","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.706","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.719","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.731","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.743","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.754","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.765","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.775","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.5963202,"monotonic":2345875.559230738,"hcas":{"mlx5_0":{"port_xmit_data":146691654396067,"port_rcv_data":148722339993411,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685297060205,"port_rcv_data":2415961306644,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147833549738585,"port_rcv_data":147227591834141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672240852984,"port_rcv_data":2290581855442,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.697","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.710","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.724","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.736","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.747","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.759","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.769","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.779","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.03,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.597124,"monotonic":2345876.560034675,"hcas":{"mlx5_0":{"port_xmit_data":146691666962635,"port_rcv_data":148723254789069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685297102086,"port_rcv_data":2416004385684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147834423518586,"port_rcv_data":147227647497295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672240895020,"port_rcv_data":2290624934482,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.678","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":143.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.691","gpu_util_pct":99.0,"memory_used_mib":79959.0,"power_w":134.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.704","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":143.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.716","gpu_util_pct":98.0,"memory_used_mib":79963.0,"power_w":138.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.727","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.738","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.748","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":140.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.758","gpu_util_pct":98.0,"memory_used_mib":80045.0,"power_w":134.79,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.5977154,"monotonic":2345877.560625822,"hcas":{"mlx5_0":{"port_xmit_data":146691678682116,"port_rcv_data":148724196513856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685297264433,"port_rcv_data":2416013227284,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147835354853428,"port_rcv_data":147227676730271,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672241211840,"port_rcv_data":2290642617682,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.697","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.712","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.725","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.737","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.749","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.760","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.770","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.779","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.5985198,"monotonic":2345878.561430114,"hcas":{"mlx5_0":{"port_xmit_data":146691690439015,"port_rcv_data":148725186998657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685297680887,"port_rcv_data":2416088816926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147836270206088,"port_rcv_data":147227751855354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672241444045,"port_rcv_data":2290706437548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:52.696","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:52.710","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.723","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.735","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.747","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.757","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.767","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.777","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.5993276,"monotonic":2345879.562237996,"hcas":{"mlx5_0":{"port_xmit_data":146691703360689,"port_rcv_data":148726079968448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685297981664,"port_rcv_data":2416103569606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147837148737186,"port_rcv_data":147227782172695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672241771545,"port_rcv_data":2290724120748,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.697","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.710","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.723","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.735","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.747","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.758","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.768","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.778","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.46,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.600135,"monotonic":2345880.563045724,"hcas":{"mlx5_0":{"port_xmit_data":146691716384698,"port_rcv_data":148727030492381,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685298364127,"port_rcv_data":2416186536710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147838018926012,"port_rcv_data":147227868900290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672241994497,"port_rcv_data":2290798271356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.681","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.694","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.708","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.02,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.720","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.731","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.742","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.752","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.762","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.600958,"monotonic":2345881.563868709,"hcas":{"mlx5_0":{"port_xmit_data":146691729192382,"port_rcv_data":148727965368602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685298582382,"port_rcv_data":2416254996122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147838885071164,"port_rcv_data":147227958810616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672242367613,"port_rcv_data":2290875489253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.701","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.714","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.727","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.738","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.750","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.761","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.771","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.782","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.6011093,"monotonic":2345882.564019778,"hcas":{"mlx5_0":{"port_xmit_data":146691741812962,"port_rcv_data":148728884202160,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685298642538,"port_rcv_data":2416313988969,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147839744525956,"port_rcv_data":147228030360174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672242428497,"port_rcv_data":2290934424357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:56.701","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.714","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.728","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.740","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.751","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.762","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.772","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.782","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.601916,"monotonic":2345883.564826624,"hcas":{"mlx5_0":{"port_xmit_data":146691754667990,"port_rcv_data":148729793882215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685298827732,"port_rcv_data":2416352367595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147840616209018,"port_rcv_data":147228090411393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672242774891,"port_rcv_data":2290981840758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.701","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.715","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.727","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.739","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":150.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.750","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.760","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.770","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.779","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.6027284,"monotonic":2345884.565638866,"hcas":{"mlx5_0":{"port_xmit_data":146691768182736,"port_rcv_data":148730623226968,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685299191765,"port_rcv_data":2416402695414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147841396354097,"port_rcv_data":147228142056752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672242962906,"port_rcv_data":2291020219872,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:58.684","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.697","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.710","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.722","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.733","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.743","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.753","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.762","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.603414,"monotonic":2345885.56632441,"hcas":{"mlx5_0":{"port_xmit_data":146691781883516,"port_rcv_data":148731416181692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685299491612,"port_rcv_data":2416432025102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147842160288409,"port_rcv_data":147228187524530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672243308137,"port_rcv_data":2291052254934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.700","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.714","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.727","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.77,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.739","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.751","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.761","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.771","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.781","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.604222,"monotonic":2345886.567132411,"hcas":{"mlx5_0":{"port_xmit_data":146691794932017,"port_rcv_data":148732307858977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685299840796,"port_rcv_data":2416449829702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147843033021028,"port_rcv_data":147228209370204,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672243491859,"port_rcv_data":2291061365040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.702","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.715","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.728","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.47,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.741","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.752","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.763","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.774","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.784","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.6050277,"monotonic":2345887.567938069,"hcas":{"mlx5_0":{"port_xmit_data":146691807799615,"port_rcv_data":148733210662376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685300017047,"port_rcv_data":2416478769061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147843909102984,"port_rcv_data":147228259934904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672243827697,"port_rcv_data":2291099207049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:01.705","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.719","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.732","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.743","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.754","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.765","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.774","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.784","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.605838,"monotonic":2345888.568748459,"hcas":{"mlx5_0":{"port_xmit_data":146691819529709,"port_rcv_data":148734146286383,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685300077962,"port_rcv_data":2416537640793,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147844784908986,"port_rcv_data":147228357008098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672244364416,"port_rcv_data":2291184605925,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.686","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.700","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.713","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.725","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.737","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.748","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.758","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.767","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.6065946,"monotonic":2345889.56950505,"hcas":{"mlx5_0":{"port_xmit_data":146691831459617,"port_rcv_data":148735126553385,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685300585742,"port_rcv_data":2416593702269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147845711230319,"port_rcv_data":147228406896315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672244554430,"port_rcv_data":2291222983241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.704","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.712","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.720","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.728","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.736","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.744","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.752","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.760","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.607432,"monotonic":2345890.57034255,"hcas":{"mlx5_0":{"port_xmit_data":146691842969119,"port_rcv_data":148736077532629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685300772656,"port_rcv_data":2416632080097,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147846623599691,"port_rcv_data":147228447794419,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672244584578,"port_rcv_data":2291252519469,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.704","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.718","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.731","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.743","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.754","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.765","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.775","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.785","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.45,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.608239,"monotonic":2345891.571149405,"hcas":{"mlx5_0":{"port_xmit_data":146691854534739,"port_rcv_data":148737039277000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685300803005,"port_rcv_data":2416661623727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147847555836922,"port_rcv_data":147228488919188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672244614911,"port_rcv_data":2291282063099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.708","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.722","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.735","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.79,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.747","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.758","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.769","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.779","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.789","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.6090431,"monotonic":2345892.571953733,"hcas":{"mlx5_0":{"port_xmit_data":146691866168070,"port_rcv_data":148737927143530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685300833680,"port_rcv_data":2416691159955,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147848412473287,"port_rcv_data":147228530075727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672244645059,"port_rcv_data":2291311599327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:06.690","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.704","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.716","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.48,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.728","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.739","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.750","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.760","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.770","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.2,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.6098678,"monotonic":2345893.572778352,"hcas":{"mlx5_0":{"port_xmit_data":146691878975762,"port_rcv_data":148738817062609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685300843522,"port_rcv_data":2416700733213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147849293301155,"port_rcv_data":147228573351489,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672245044881,"port_rcv_data":2291342104953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.726","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.739","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.753","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.765","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.777","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.788","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.798","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.808","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.6106532,"monotonic":2345894.573563754,"hcas":{"mlx5_0":{"port_xmit_data":146691892282780,"port_rcv_data":148739749010692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685301352651,"port_rcv_data":2416752436913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147850173902615,"port_rcv_data":147228616909692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672245218528,"port_rcv_data":2291372878432,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.726","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.739","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.753","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.765","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.776","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.788","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.798","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.808","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.6114414,"monotonic":2345895.574351919,"hcas":{"mlx5_0":{"port_xmit_data":146691905183869,"port_rcv_data":148740688736406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685301569279,"port_rcv_data":2416820065103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147851048177885,"port_rcv_data":147228706122437,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672245594139,"port_rcv_data":2291449348547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.710","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.723","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.736","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.98,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.749","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.760","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.771","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.781","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.790","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.36,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.6122465,"monotonic":2345896.57515691,"hcas":{"mlx5_0":{"port_xmit_data":146691917258727,"port_rcv_data":148741661707744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685301756968,"port_rcv_data":2416858334674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147851981649491,"port_rcv_data":147228768740102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672245988227,"port_rcv_data":2291500065598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.713","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.726","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.739","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.751","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.763","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.774","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.784","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.793","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.45,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.6130502,"monotonic":2345897.575960544,"hcas":{"mlx5_0":{"port_xmit_data":146691929501217,"port_rcv_data":148742668688822,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685302253681,"port_rcv_data":2416914316092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147852932947602,"port_rcv_data":147228815226417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672246127510,"port_rcv_data":2291534699713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.712","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.725","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.738","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.750","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.761","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.772","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.782","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.792","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.6138558,"monotonic":2345898.57676651,"hcas":{"mlx5_0":{"port_xmit_data":146691940888506,"port_rcv_data":148743614401389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685302284015,"port_rcv_data":2416943743839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147853850206775,"port_rcv_data":147228856078750,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672246157549,"port_rcv_data":2291564127460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.730","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.744","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.756","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.768","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.779","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.790","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.800","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.811","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.6146467,"monotonic":2345899.577557383,"hcas":{"mlx5_0":{"port_xmit_data":146691952224059,"port_rcv_data":148744543307773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685302300305,"port_rcv_data":2416958783035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147854763648836,"port_rcv_data":147228900164598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672246536927,"port_rcv_data":2291596907599,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:13.729","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.743","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.756","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.768","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.779","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.790","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.799","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.22,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.809","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.17,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.6154394,"monotonic":2345900.578350006,"hcas":{"mlx5_0":{"port_xmit_data":146691963874233,"port_rcv_data":148745488862901,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685302469488,"port_rcv_data":2416978029747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147855690490233,"port_rcv_data":147228939679580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672246868565,"port_rcv_data":2291624997083,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.729","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.743","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.767","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.778","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.789","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.799","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.809","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.616233,"monotonic":2345901.579143638,"hcas":{"mlx5_0":{"port_xmit_data":146691975916203,"port_rcv_data":148746507736837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685303023117,"port_rcv_data":2417067977153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147856619888132,"port_rcv_data":147229017080883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672246942147,"port_rcv_data":2291690911350,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.717","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.731","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.744","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.756","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.768","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.778","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.788","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.798","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.6170366,"monotonic":2345902.579946923,"hcas":{"mlx5_0":{"port_xmit_data":146691987838882,"port_rcv_data":148747509331289,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685303229515,"port_rcv_data":2417120460155,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147857571404496,"port_rcv_data":147229096994108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672247446005,"port_rcv_data":2291758938964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:16.733","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.746","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.759","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.771","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.782","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.793","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.804","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.814","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.6178257,"monotonic":2345903.580736297,"hcas":{"mlx5_0":{"port_xmit_data":146691999925355,"port_rcv_data":148748533651999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685303606351,"port_rcv_data":2417197385010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147858518154019,"port_rcv_data":147229176583623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672247662292,"port_rcv_data":2291826639803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.716","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.730","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.742","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.755","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.766","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.776","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.787","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.797","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.6186333,"monotonic":2345904.58154388,"hcas":{"mlx5_0":{"port_xmit_data":146692011322714,"port_rcv_data":148749521999844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685303802891,"port_rcv_data":2417250074668,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147859455169464,"port_rcv_data":147229231640964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672247706932,"port_rcv_data":2291870427462,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.716","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.729","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.743","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.15,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.755","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.766","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.776","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.786","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.796","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.6194427,"monotonic":2345905.582353315,"hcas":{"mlx5_0":{"port_xmit_data":146692022116778,"port_rcv_data":148750580995125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685303859286,"port_rcv_data":2417307513388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147860457103752,"port_rcv_data":147229308595100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672247952995,"port_rcv_data":2291936561326,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.716","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.729","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.742","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.754","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.766","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.777","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.787","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.797","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.6202526,"monotonic":2345906.583163274,"hcas":{"mlx5_0":{"port_xmit_data":146692032813156,"port_rcv_data":148751583188492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685304056926,"port_rcv_data":2417330775506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147861434579864,"port_rcv_data":147229351538511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672248321817,"port_rcv_data":2291968933274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.701","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.715","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.728","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.740","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.752","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.763","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.773","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.783","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.621077,"monotonic":2345907.583987415,"hcas":{"mlx5_0":{"port_xmit_data":146692044516797,"port_rcv_data":148752538785627,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685304551221,"port_rcv_data":2417357275181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147862365240122,"port_rcv_data":147229371753892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672248475763,"port_rcv_data":2291977836130,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:21.720","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.734","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.747","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.759","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.771","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.782","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.74,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.792","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.802","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.13,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.6218822,"monotonic":2345908.584792617,"hcas":{"mlx5_0":{"port_xmit_data":146692055678089,"port_rcv_data":148753507780303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685304599535,"port_rcv_data":2417396720133,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147863295289552,"port_rcv_data":147229448551428,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672248997416,"port_rcv_data":2292043715205,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.720","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.734","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.747","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.759","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.770","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.781","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.791","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.801","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.6226892,"monotonic":2345909.585599746,"hcas":{"mlx5_0":{"port_xmit_data":146692066833332,"port_rcv_data":148754538769724,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685305074393,"port_rcv_data":2417432818053,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147864290164993,"port_rcv_data":147229468849828,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672249006871,"port_rcv_data":2292053288325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.738","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.752","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.765","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.777","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.789","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.799","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.811","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.821","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.6234763,"monotonic":2345910.586386755,"hcas":{"mlx5_0":{"port_xmit_data":146692077510132,"port_rcv_data":148755553899696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685305258951,"port_rcv_data":2417470379013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147865268370271,"port_rcv_data":147229528694094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672249373213,"port_rcv_data":2292102579165,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.720","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.734","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.747","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.759","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.771","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.782","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.792","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.802","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.6242807,"monotonic":2345911.587191244,"hcas":{"mlx5_0":{"port_xmit_data":146692088649944,"port_rcv_data":148756594143077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685305592604,"port_rcv_data":2417502421893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147866275233051,"port_rcv_data":147229568689894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672249681865,"port_rcv_data":2292131733765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.705","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.719","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.732","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.743","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.754","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.765","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.775","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.785","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.6251051,"monotonic":2345912.588015606,"hcas":{"mlx5_0":{"port_xmit_data":146692099849009,"port_rcv_data":148757641186861,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685305917670,"port_rcv_data":2417534464773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147867290508668,"port_rcv_data":147229602834828,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672249850071,"port_rcv_data":2292154935045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.745","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.758","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.772","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.784","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.795","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.806","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.816","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.826","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.18,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.625889,"monotonic":2345913.588799684,"hcas":{"mlx5_0":{"port_xmit_data":146692110660010,"port_rcv_data":148758654079267,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685306140591,"port_rcv_data":2417561743829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147868278490216,"port_rcv_data":147229640917962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672250074573,"port_rcv_data":2292182215401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.724","gpu_util_pct":99.0,"memory_used_mib":79997.0,"power_w":143.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.738","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.751","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.764","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.776","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.787","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.797","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.807","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.6266952,"monotonic":2345914.589605646,"hcas":{"mlx5_0":{"port_xmit_data":146692120692659,"port_rcv_data":148759672864676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685306154634,"port_rcv_data":2417576103509,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147869282940767,"port_rcv_data":147229665362776,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672250088585,"port_rcv_data":2292196575081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.724","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.738","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.751","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.762","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.773","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.783","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.794","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.803","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.6275027,"monotonic":2345915.590413287,"hcas":{"mlx5_0":{"port_xmit_data":146692130152493,"port_rcv_data":148760650797408,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685306349578,"port_rcv_data":2417602147222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147870233972030,"port_rcv_data":147229725551608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672250677864,"port_rcv_data":2292247516855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.724","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.738","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.750","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.762","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.773","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.784","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.794","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.804","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.6283107,"monotonic":2345916.591221332,"hcas":{"mlx5_0":{"port_xmit_data":146692139980101,"port_rcv_data":148761710991989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685306915219,"port_rcv_data":2417708301998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147871190111464,"port_rcv_data":147229816178247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672250914332,"port_rcv_data":2292328794619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.708","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.722","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.735","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.747","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.758","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":132.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.769","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.779","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.789","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.92,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.6291091,"monotonic":2345917.592019876,"hcas":{"mlx5_0":{"port_xmit_data":146692150058915,"port_rcv_data":148762745907545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307062361,"port_rcv_data":2417716871520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147872215229590,"port_rcv_data":147229851849413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251367598,"port_rcv_data":2292354503283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.729","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.742","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.767","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.777","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.789","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.799","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.809","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.6299143,"monotonic":2345918.592824881,"hcas":{"mlx5_0":{"port_xmit_data":146692160212654,"port_rcv_data":148763771973736,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307502375,"port_rcv_data":2417742579534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147873215356810,"port_rcv_data":147229870187166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251511299,"port_rcv_data":2292363073455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.728","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.742","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.767","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.778","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.789","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.799","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.809","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.6307235,"monotonic":2345919.593634012,"hcas":{"mlx5_0":{"port_xmit_data":146692168362151,"port_rcv_data":148764630959447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307502375,"port_rcv_data":2417742579534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147874074745738,"port_rcv_data":147229878391556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251511299,"port_rcv_data":2292363073455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.728","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.04,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.742","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.767","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.778","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.790","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.56,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.800","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.810","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":125.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.6315308,"monotonic":2345920.594441375,"hcas":{"mlx5_0":{"port_xmit_data":146692176257209,"port_rcv_data":148765515471444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307502375,"port_rcv_data":2417742579534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147874961056064,"port_rcv_data":147229886362598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251511299,"port_rcv_data":2292363073455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.728","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.741","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.767","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.778","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":126.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.789","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.799","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.808","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.63234,"monotonic":2345921.595250578,"hcas":{"mlx5_0":{"port_xmit_data":146692183879235,"port_rcv_data":148766366540432,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307502375,"port_rcv_data":2417742579534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147875812101334,"port_rcv_data":147229894023856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251511299,"port_rcv_data":2292363073455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.732","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.745","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.759","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.771","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.783","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.793","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.803","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.813","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.6331449,"monotonic":2345922.596055572,"hcas":{"mlx5_0":{"port_xmit_data":146692193181352,"port_rcv_data":148767349276620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307511830,"port_rcv_data":2417752152654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147876785583084,"port_rcv_data":147229912957284,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251520676,"port_rcv_data":2292372646575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.749","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.762","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.775","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.787","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.799","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.809","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.829","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.6339343,"monotonic":2345923.596844847,"hcas":{"mlx5_0":{"port_xmit_data":146692202927505,"port_rcv_data":148768369008203,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307525780,"port_rcv_data":2417766512334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147877791048148,"port_rcv_data":147229937124631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251534673,"port_rcv_data":2292387006255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.732","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.745","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.758","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.77,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.771","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.782","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.794","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.804","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.814","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.89,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.6347413,"monotonic":2345924.597651781,"hcas":{"mlx5_0":{"port_xmit_data":146692212654386,"port_rcv_data":148769385708219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307542861,"port_rcv_data":2417784230297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147878791500143,"port_rcv_data":147229975633812,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251562976,"port_rcv_data":2292415725615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.732","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.746","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.759","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.771","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.782","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.793","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.803","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.813","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.69,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.6355479,"monotonic":2345925.598458483,"hcas":{"mlx5_0":{"port_xmit_data":146692221886362,"port_rcv_data":148770358967918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307568327,"port_rcv_data":2417809591374,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147879736512596,"port_rcv_data":147229999239835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251577019,"port_rcv_data":2292430085295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.749","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.763","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.776","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.787","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.798","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.809","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.828","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.6363416,"monotonic":2345926.599252203,"hcas":{"mlx5_0":{"port_xmit_data":146692231084959,"port_rcv_data":148771316760456,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307582308,"port_rcv_data":2417823951054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147880680078928,"port_rcv_data":147230022845862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251590984,"port_rcv_data":2292444444975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.736","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.750","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.763","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.775","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.786","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.796","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.806","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.816","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.637148,"monotonic":2345927.600058463,"hcas":{"mlx5_0":{"port_xmit_data":146692241816439,"port_rcv_data":148772487901740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307596320,"port_rcv_data":2417838310734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147881836423125,"port_rcv_data":147230047979470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251604996,"port_rcv_data":2292458804655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.753","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.767","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.780","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.792","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.803","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.833","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.637937,"monotonic":2345928.600847891,"hcas":{"mlx5_0":{"port_xmit_data":146692252824027,"port_rcv_data":148773684631654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685307596320,"port_rcv_data":2417838310734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147883035882959,"port_rcv_data":147230076766199,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672251954196,"port_rcv_data":2292476487855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.754","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.767","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.780","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.792","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.803","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.834","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.6387277,"monotonic":2345929.601638359,"hcas":{"mlx5_0":{"port_xmit_data":146692264284517,"port_rcv_data":148775000448116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308013937,"port_rcv_data":2417955121542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147884235529101,"port_rcv_data":147230204544028,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252387886,"port_rcv_data":2292593299355,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.738","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.752","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.764","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.776","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.787","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.798","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.807","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.817","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.67,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.6395354,"monotonic":2345930.602446105,"hcas":{"mlx5_0":{"port_xmit_data":146692275602623,"port_rcv_data":148776214811177,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308328757,"port_rcv_data":2417972804811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147885432731182,"port_rcv_data":147230232990907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252687873,"port_rcv_data":2292610603981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.736","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.750","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.763","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.775","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.786","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.797","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.806","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.816","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.73,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.6403437,"monotonic":2345931.603254213,"hcas":{"mlx5_0":{"port_xmit_data":146692286901805,"port_rcv_data":148777430430373,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308580074,"port_rcv_data":2417988767371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147886633081902,"port_rcv_data":147230261894947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992417,"port_rcv_data":2292628323337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.741","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.755","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.768","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.779","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.792","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.805","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.817","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.828","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.6411388,"monotonic":2345932.604049319,"hcas":{"mlx5_0":{"port_xmit_data":146692298204440,"port_rcv_data":148778639751064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147887821207918,"port_rcv_data":147230272891510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.722","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.736","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.749","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.761","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.772","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.783","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.793","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.803","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.6419616,"monotonic":2345933.604871896,"hcas":{"mlx5_0":{"port_xmit_data":146692308777659,"port_rcv_data":148779880276837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147889063802668,"port_rcv_data":147230283570838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.740","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.754","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.767","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.780","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.792","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.802","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.813","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.822","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.642766,"monotonic":2345934.605676661,"hcas":{"mlx5_0":{"port_xmit_data":146692319269385,"port_rcv_data":148781135583391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147890318249193,"port_rcv_data":147230294151532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.758","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.771","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.784","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.796","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.808","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.819","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.829","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.839","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.643554,"monotonic":2345935.606464642,"hcas":{"mlx5_0":{"port_xmit_data":146692329658512,"port_rcv_data":148782371932681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147891553071354,"port_rcv_data":147230304602658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.758","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.771","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.785","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.796","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.808","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.829","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.839","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.6443505,"monotonic":2345936.607260795,"hcas":{"mlx5_0":{"port_xmit_data":146692340131532,"port_rcv_data":148783623746868,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147892807407313,"port_rcv_data":147230315195036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.726","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.739","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.752","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.764","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":150.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.775","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.785","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.795","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.804","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.6451101,"monotonic":2345937.608020413,"hcas":{"mlx5_0":{"port_xmit_data":146692350660170,"port_rcv_data":148784881081278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147894065084198,"port_rcv_data":147230325814220,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.765","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.779","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.792","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.805","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.817","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.828","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.838","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.848","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.6459026,"monotonic":2345938.608812905,"hcas":{"mlx5_0":{"port_xmit_data":146692361132427,"port_rcv_data":148786137511923,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147895321121374,"port_rcv_data":147230336380822,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.762","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.775","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.789","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.801","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.813","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:52.824","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:52.834","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:52.844","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.6463244,"monotonic":2345939.609234796,"hcas":{"mlx5_0":{"port_xmit_data":146692371696426,"port_rcv_data":148787391303454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147896573397393,"port_rcv_data":147230347013004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.749","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.762","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.776","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.788","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.799","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.810","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.820","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.830","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.647129,"monotonic":2345940.610039481,"hcas":{"mlx5_0":{"port_xmit_data":146692382198496,"port_rcv_data":148788645565788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147897829767670,"port_rcv_data":147230357624904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.728","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.742","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.754","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.766","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":150.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.776","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.787","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.797","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:54.806","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.6479576,"monotonic":2345941.610867844,"hcas":{"mlx5_0":{"port_xmit_data":146692392697142,"port_rcv_data":148789898975147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147899082885386,"port_rcv_data":147230368205667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.729","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.742","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.755","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.767","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.778","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.789","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.800","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.810","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.6487823,"monotonic":2345942.611692686,"hcas":{"mlx5_0":{"port_xmit_data":146692403136592,"port_rcv_data":148791151786587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147900335442949,"port_rcv_data":147230378726002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.748","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.762","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.774","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.786","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.797","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.807","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.817","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.826","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.19,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.6491115,"monotonic":2345943.612022124,"hcas":{"mlx5_0":{"port_xmit_data":146692413558622,"port_rcv_data":148792385328866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147901569998217,"port_rcv_data":147230389243837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.748","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.762","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.775","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.787","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.798","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.809","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.829","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.6499212,"monotonic":2345944.612831759,"hcas":{"mlx5_0":{"port_xmit_data":146692424070921,"port_rcv_data":148793642017515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147902826643731,"port_rcv_data":147230399855020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.731","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.744","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.757","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.769","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.780","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.791","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.801","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.811","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.6500137,"monotonic":2345945.612924065,"hcas":{"mlx5_0":{"port_xmit_data":146692434569196,"port_rcv_data":148794895113043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147904079426792,"port_rcv_data":147230410436748,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.748","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.763","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.776","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.787","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.798","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.809","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.829","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.6508214,"monotonic":2345946.61373177,"hcas":{"mlx5_0":{"port_xmit_data":146692445100324,"port_rcv_data":148796151031377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147905334056290,"port_rcv_data":147230421043887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.748","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.762","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.775","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.787","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.799","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.810","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.821","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.831","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.6516268,"monotonic":2345947.614537356,"hcas":{"mlx5_0":{"port_xmit_data":146692455666295,"port_rcv_data":148797411964637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147906594308535,"port_rcv_data":147230431686675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.732","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.746","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.760","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.772","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.783","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.794","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.804","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.813","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.6524518,"monotonic":2345948.615362332,"hcas":{"mlx5_0":{"port_xmit_data":146692466151023,"port_rcv_data":148798661413302,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147907846577242,"port_rcv_data":147230442303572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.752","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.766","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.780","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.792","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.803","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.833","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.78,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.6532583,"monotonic":2345949.616168863,"hcas":{"mlx5_0":{"port_xmit_data":146692476677845,"port_rcv_data":148799918586794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308929212,"port_rcv_data":2418007828118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147909103069368,"port_rcv_data":147230452911977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672252992556,"port_rcv_data":2292628324031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.753","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.766","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.780","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.792","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.804","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.815","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.825","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.835","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.6540635,"monotonic":2345950.616973742,"hcas":{"mlx5_0":{"port_xmit_data":146692487441535,"port_rcv_data":148801127540935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308938775,"port_rcv_data":2418017401238,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147910303160345,"port_rcv_data":147230473322282,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253001965,"port_rcv_data":2292637897151,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.770","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.783","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.796","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.808","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.819","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.850","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.6548522,"monotonic":2345951.617762653,"hcas":{"mlx5_0":{"port_xmit_data":146692498171779,"port_rcv_data":148802319217909,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308962242,"port_rcv_data":2418041334038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147911471297569,"port_rcv_data":147230508034917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253025339,"port_rcv_data":2292661829951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.752","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.766","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.779","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.792","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.803","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.815","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.825","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.835","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.6556594,"monotonic":2345952.618569927,"hcas":{"mlx5_0":{"port_xmit_data":146692509067304,"port_rcv_data":148803522322923,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685308990297,"port_rcv_data":2418070053398,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147912643187581,"port_rcv_data":147230547658948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253053285,"port_rcv_data":2292690549311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.752","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.767","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.780","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.792","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.803","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.834","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.6564667,"monotonic":2345953.619377218,"hcas":{"mlx5_0":{"port_xmit_data":146692519897819,"port_rcv_data":148804725443979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685309018352,"port_rcv_data":2418098772758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147913820049585,"port_rcv_data":147230587271484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253083634,"port_rcv_data":2292722473216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.757","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.771","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.784","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.795","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.806","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.817","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.827","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:07.836","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.6571126,"monotonic":2345954.62002301,"hcas":{"mlx5_0":{"port_xmit_data":146692530738479,"port_rcv_data":148805954964734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685309074447,"port_rcv_data":2418156211478,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147914992275532,"port_rcv_data":147230655564313,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253137497,"port_rcv_data":2292776707391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.738","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.751","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.764","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.776","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.788","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.798","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.809","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.51,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.818","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.6579368,"monotonic":2345955.620847464,"hcas":{"mlx5_0":{"port_xmit_data":146692541564104,"port_rcv_data":148807156699814,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685309102905,"port_rcv_data":2418184930838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147916164393961,"port_rcv_data":147230695134705,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253165629,"port_rcv_data":2292805426751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.756","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.770","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.783","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.795","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.806","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.816","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.826","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.836","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.85,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.6587455,"monotonic":2345956.621656235,"hcas":{"mlx5_0":{"port_xmit_data":146692552449757,"port_rcv_data":148808352437784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685309116901,"port_rcv_data":2418199290518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147917346803169,"port_rcv_data":147230729281110,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253352885,"port_rcv_data":2292828628031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.756","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.770","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.783","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.795","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.807","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.828","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.838","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.659552,"monotonic":2345957.62246251,"hcas":{"mlx5_0":{"port_xmit_data":146692563424833,"port_rcv_data":148809535440233,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685309294345,"port_rcv_data":2418208132118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147918520679150,"port_rcv_data":147230757836452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253682663,"port_rcv_data":2292846311231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.756","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.771","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.784","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.796","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.807","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.819","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.829","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.839","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.6603582,"monotonic":2345958.623268677,"hcas":{"mlx5_0":{"port_xmit_data":146692574931924,"port_rcv_data":148810741921704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685309671662,"port_rcv_data":2418271763493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147919662140814,"port_rcv_data":147230823675387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672253909412,"port_rcv_data":2292901242108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.761","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.775","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.788","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":158.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.800","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.812","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.822","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.833","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.842","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.55,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.661171,"monotonic":2345959.6240815,"hcas":{"mlx5_0":{"port_xmit_data":146692586422755,"port_rcv_data":148811886493515,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685309853957,"port_rcv_data":2418304537893,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147920776010534,"port_rcv_data":147230876659592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672254254365,"port_rcv_data":2292942858108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.797","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.811","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.837","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.848","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.869","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.879","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.6619384,"monotonic":2345960.624849177,"hcas":{"mlx5_0":{"port_xmit_data":146692598595886,"port_rcv_data":148812912704752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685310183642,"port_rcv_data":2418336580773,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147921769000343,"port_rcv_data":147230920557702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672254584685,"port_rcv_data":2292974900988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.761","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.775","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.788","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.800","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.812","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.823","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.833","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.843","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.08,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.6627493,"monotonic":2345961.625659564,"hcas":{"mlx5_0":{"port_xmit_data":146692611247940,"port_rcv_data":148813892757211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685310514800,"port_rcv_data":2418368623791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147922717136071,"port_rcv_data":147230957011561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672254763865,"port_rcv_data":2292999902965,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.760","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.774","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.787","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.800","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.811","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.822","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.832","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.842","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.6635547,"monotonic":2345962.626465006,"hcas":{"mlx5_0":{"port_xmit_data":146692623954137,"port_rcv_data":148814811595344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685310783833,"port_rcv_data":2418398406021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147923607162294,"port_rcv_data":147230989616723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672254870707,"port_rcv_data":2293019044336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.761","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.81,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.775","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.788","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":148.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.800","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.811","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.822","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.832","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.842","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.6643636,"monotonic":2345963.627274018,"hcas":{"mlx5_0":{"port_xmit_data":146692636502359,"port_rcv_data":148815724120665,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685310797845,"port_rcv_data":2418412765701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147924506467357,"port_rcv_data":147231016557867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672254884688,"port_rcv_data":2293033404016,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.765","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.779","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.792","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.805","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.816","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.838","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.847","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.665046,"monotonic":2345964.627956498,"hcas":{"mlx5_0":{"port_xmit_data":146692648920395,"port_rcv_data":148816624395506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685310812214,"port_rcv_data":2418427125381,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147925391796783,"port_rcv_data":147231043313791,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672254898793,"port_rcv_data":2293047763696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.782","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.796","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.809","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.820","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.832","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.11,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.843","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.853","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.863","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.6658382,"monotonic":2345965.628748861,"hcas":{"mlx5_0":{"port_xmit_data":146692661357079,"port_rcv_data":148817514701001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685310812338,"port_rcv_data":2418427125881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147926281292259,"port_rcv_data":147231089961367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672255409549,"port_rcv_data":2293082023717,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.764","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.778","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.791","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.02,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.803","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.814","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.824","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.834","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.844","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.6666498,"monotonic":2345966.629560415,"hcas":{"mlx5_0":{"port_xmit_data":146692674485580,"port_rcv_data":148818521385087,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685311388783,"port_rcv_data":2418533130995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147927184237579,"port_rcv_data":147231183204646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672255649349,"port_rcv_data":2293162609054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.747","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.760","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.773","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.785","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.795","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.805","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.815","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.825","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.667484,"monotonic":2345967.630394733,"hcas":{"mlx5_0":{"port_xmit_data":146692688027075,"port_rcv_data":148819332808925,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685311586346,"port_rcv_data":2418580873875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147927946053914,"port_rcv_data":147231235412607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672255688394,"port_rcv_data":2293201509162,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.782","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.796","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.809","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.821","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.832","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.843","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.853","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:21.862","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.33,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.668278,"monotonic":2345968.631188545,"hcas":{"mlx5_0":{"port_xmit_data":146692701255793,"port_rcv_data":148820154363550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685311668139,"port_rcv_data":2418662828941,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147928687460609,"port_rcv_data":147231330607318,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672255770776,"port_rcv_data":2293283465528,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.749","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.763","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.776","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.788","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.799","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.809","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.829","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.669104,"monotonic":2345969.632014693,"hcas":{"mlx5_0":{"port_xmit_data":146692714086158,"port_rcv_data":148820858515243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685311710346,"port_rcv_data":2418705908050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147929348209556,"port_rcv_data":147231395323938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672255968277,"port_rcv_data":2293335386196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.769","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.783","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.796","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.808","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.819","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":131.01,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:23.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:23.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:23.850","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.6699107,"monotonic":2345970.632821363,"hcas":{"mlx5_0":{"port_xmit_data":146692726722243,"port_rcv_data":148821467821037,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685311917534,"port_rcv_data":2418767228027,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147929896708215,"port_rcv_data":147231486916556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672256490410,"port_rcv_data":2293414596470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.755","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.769","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.782","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.793","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.805","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.815","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.825","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.834","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":125.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.6707332,"monotonic":2345971.633643758,"hcas":{"mlx5_0":{"port_xmit_data":146692739640321,"port_rcv_data":148822097477448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685312470683,"port_rcv_data":2418851448077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147930442839675,"port_rcv_data":147231556715736,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672256546551,"port_rcv_data":2293472085964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.769","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.783","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.796","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.78,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.808","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:25.820","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.53,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:25.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:25.841","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:25.852","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.6715364,"monotonic":2345972.634446987,"hcas":{"mlx5_0":{"port_xmit_data":146692752124729,"port_rcv_data":148822724373616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685312512843,"port_rcv_data":2418894477269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147931025501116,"port_rcv_data":147231629938828,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672256931075,"port_rcv_data":2293532798356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.752","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.766","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.779","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.5,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.790","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.802","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.812","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.822","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.832","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.672364,"monotonic":2345973.635274531,"hcas":{"mlx5_0":{"port_xmit_data":146692764721721,"port_rcv_data":148823310323121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685312871900,"port_rcv_data":2418940829981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147931566918535,"port_rcv_data":147231679843565,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672257121260,"port_rcv_data":2293570309468,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.772","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.786","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.799","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.13,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.810","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.821","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.39,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.832","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.45,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.842","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.851","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.6724892,"monotonic":2345974.635399576,"hcas":{"mlx5_0":{"port_xmit_data":146692777415412,"port_rcv_data":148823904211991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685313265120,"port_rcv_data":2418992037319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147932108333957,"port_rcv_data":147231734526719,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672257310469,"port_rcv_data":2293612675206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.773","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.786","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.800","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.3,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.812","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.823","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.78,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.834","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.4,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.854","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.6731114,"monotonic":2345975.636022113,"hcas":{"mlx5_0":{"port_xmit_data":146692789768004,"port_rcv_data":148824484048306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685313482321,"port_rcv_data":2419038186207,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147932642494814,"port_rcv_data":147231810398142,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672257877815,"port_rcv_data":2293676509638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.754","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.768","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.780","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.792","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.803","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.43,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.813","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.823","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.833","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.38,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.6739376,"monotonic":2345976.636848123,"hcas":{"mlx5_0":{"port_xmit_data":146692802350042,"port_rcv_data":148825079602272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685313866256,"port_rcv_data":2419094919123,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147933181626877,"port_rcv_data":147231870577363,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672258073642,"port_rcv_data":2293724402126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.754","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.768","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.782","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.58,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.794","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.805","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.817","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.827","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.837","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.6747599,"monotonic":2345977.637670373,"hcas":{"mlx5_0":{"port_xmit_data":146692814684445,"port_rcv_data":148825659013597,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685314076607,"port_rcv_data":2419139835685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147933716385655,"port_rcv_data":147231939129705,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672258473573,"port_rcv_data":2293780836376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.755","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.03,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.769","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.782","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.794","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.805","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.63,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.817","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.827","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.837","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.32,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.6755824,"monotonic":2345978.638492982,"hcas":{"mlx5_0":{"port_xmit_data":146692827277721,"port_rcv_data":148826274272348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685314449258,"port_rcv_data":2419217060153,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147934254919279,"port_rcv_data":147232025963206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672258841745,"port_rcv_data":2293855384840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.773","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.5,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.787","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.800","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.813","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.824","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.23,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.836","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.846","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.856","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":128.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.6763895,"monotonic":2345979.639299925,"hcas":{"mlx5_0":{"port_xmit_data":146692839913305,"port_rcv_data":148826940072940,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685314871865,"port_rcv_data":2419344923405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147934794122194,"port_rcv_data":147232157201314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672259113243,"port_rcv_data":2293974390696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.757","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.771","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.784","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.796","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.807","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":128.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.828","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:33.838","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":126.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.6772165,"monotonic":2345980.640127111,"hcas":{"mlx5_0":{"port_xmit_data":146692852359390,"port_rcv_data":148827546181270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685315084370,"port_rcv_data":2419411466348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147935333429633,"port_rcv_data":147232244900268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672259485940,"port_rcv_data":2294049757663,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.778","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.792","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.805","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.03,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.817","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.828","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.55,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.839","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.83,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.849","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.859","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":127.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.678018,"monotonic":2345981.640928622,"hcas":{"mlx5_0":{"port_xmit_data":146692864947365,"port_rcv_data":148828156725881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685315540025,"port_rcv_data":2419487976171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147935867227711,"port_rcv_data":147232318863169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672259695841,"port_rcv_data":2294111513598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.759","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.772","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.785","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.797","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.809","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.820","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.67,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.830","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.840","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.678842,"monotonic":2345982.641752595,"hcas":{"mlx5_0":{"port_xmit_data":146692877578332,"port_rcv_data":148828733636019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685315936872,"port_rcv_data":2419528462272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147936409227133,"port_rcv_data":147232381155760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672260013343,"port_rcv_data":2294161607718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.760","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.773","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.786","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.67,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.798","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.809","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.8,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.820","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.830","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.840","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":126.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.6796656,"monotonic":2345983.642576162,"hcas":{"mlx5_0":{"port_xmit_data":146692890397580,"port_rcv_data":148829499814321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685316049572,"port_rcv_data":2419619152187,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147937081825705,"port_rcv_data":147232504514255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672260711820,"port_rcv_data":2294272189518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.794","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.807","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.820","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.96,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.832","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.844","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.855","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.08,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.865","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.875","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.680455,"monotonic":2345984.643365492,"hcas":{"mlx5_0":{"port_xmit_data":146692904297789,"port_rcv_data":148830317931225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685316699379,"port_rcv_data":2419694905120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147937822838205,"port_rcv_data":147232569973542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672260939685,"port_rcv_data":2294324464906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.798","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.811","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.837","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.847","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.858","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.868","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.878","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.6812444,"monotonic":2345985.644154725,"hcas":{"mlx5_0":{"port_xmit_data":146692917749607,"port_rcv_data":148831100346181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685317069240,"port_rcv_data":2419741806061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147938558893198,"port_rcv_data":147232638901629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672261478016,"port_rcv_data":2294380280331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.750","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.87,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.764","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.776","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.788","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.798","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.809","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:39.828","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.6820848,"monotonic":2345986.644995433,"hcas":{"mlx5_0":{"port_xmit_data":146692931236655,"port_rcv_data":148831908419275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685317476223,"port_rcv_data":2419814433580,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147939294917217,"port_rcv_data":147232731436244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672261983254,"port_rcv_data":2294459739705,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.798","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.47,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.812","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.96,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.836","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.847","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.858","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.99,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.868","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.877","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.6828763,"monotonic":2345987.645786731,"hcas":{"mlx5_0":{"port_xmit_data":146692944306359,"port_rcv_data":148832822699312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685318078212,"port_rcv_data":2419891613253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147940134557649,"port_rcv_data":147232796421011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672262174090,"port_rcv_data":2294512261056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.764","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.777","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.790","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.55,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.802","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.814","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.825","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.835","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.845","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.6836991,"monotonic":2345988.646609715,"hcas":{"mlx5_0":{"port_xmit_data":146692957044323,"port_rcv_data":148833756193189,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685318448523,"port_rcv_data":2419939283903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147941019225502,"port_rcv_data":147232865342547,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672262729222,"port_rcv_data":2294568805645,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.782","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.796","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.809","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.821","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.832","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.843","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:42.853","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:42.863","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.6845057,"monotonic":2345989.647416116,"hcas":{"mlx5_0":{"port_xmit_data":146692969344000,"port_rcv_data":148834822260236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685318902924,"port_rcv_data":2420059020180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147941967460793,"port_rcv_data":147233005567981,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672263326887,"port_rcv_data":2294697416852,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.781","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.795","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.808","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.819","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.830","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.840","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.97,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.860","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.685317,"monotonic":2345990.648227701,"hcas":{"mlx5_0":{"port_xmit_data":146692981297794,"port_rcv_data":148835865806163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685319260664,"port_rcv_data":2420122564263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147942947166249,"port_rcv_data":147233080907120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672263694950,"port_rcv_data":2294761026754,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.801","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.815","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.828","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.840","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.851","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.861","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.882","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.6861072,"monotonic":2345991.649017648,"hcas":{"mlx5_0":{"port_xmit_data":146692993234509,"port_rcv_data":148836868108067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685319568230,"port_rcv_data":2420140247670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147943931161493,"port_rcv_data":147233118669139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672264151936,"port_rcv_data":2294787137193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.785","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.799","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.812","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.824","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.835","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:45.846","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:45.856","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:45.866","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.6869142,"monotonic":2345992.649824877,"hcas":{"mlx5_0":{"port_xmit_data":146693005317631,"port_rcv_data":148837874223673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685320157850,"port_rcv_data":2420175061007,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147944902686037,"port_rcv_data":147233138895120,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672264303898,"port_rcv_data":2294795841598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.785","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.798","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.811","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.823","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.834","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.845","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.855","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.864","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.41,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.6877227,"monotonic":2345993.650633139,"hcas":{"mlx5_0":{"port_xmit_data":146693017055971,"port_rcv_data":148838861977377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685320418064,"port_rcv_data":2420191587941,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147945875660224,"port_rcv_data":147233177579673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672264765643,"port_rcv_data":2294822988378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.768","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.782","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.794","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.33,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.806","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.817","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.837","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.847","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.6885524,"monotonic":2345994.651462999,"hcas":{"mlx5_0":{"port_xmit_data":146693028995137,"port_rcv_data":148839862244091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685320776021,"port_rcv_data":2420211256100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147946857807554,"port_rcv_data":147233207308185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265075721,"port_rcv_data":2294841086772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.788","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.802","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.815","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.827","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.838","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.858","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.867","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.6891139,"monotonic":2345995.652024183,"hcas":{"mlx5_0":{"port_xmit_data":146693040573709,"port_rcv_data":148840905758688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321079883,"port_rcv_data":2420229146594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147947884083407,"port_rcv_data":147233245766811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537481,"port_rcv_data":2294868233552,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.806","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.820","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.833","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.845","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.856","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.877","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.887","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.16,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.6899123,"monotonic":2345996.652822792,"hcas":{"mlx5_0":{"port_xmit_data":146693052045632,"port_rcv_data":148842017896467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147948977802294,"port_rcv_data":147233256971562,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.771","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.785","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.798","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.809","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.821","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.832","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:50.841","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:50.851","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.690737,"monotonic":2345997.653647421,"hcas":{"mlx5_0":{"port_xmit_data":146693062886321,"port_rcv_data":148843146440002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147950105881238,"port_rcv_data":147233267860851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.806","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.819","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.832","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:51.844","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:51.856","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:51.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.877","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.887","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.6915278,"monotonic":2345998.65443845,"hcas":{"mlx5_0":{"port_xmit_data":146693073630074,"port_rcv_data":148844312829910,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147951272050791,"port_rcv_data":147233278663409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.788","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.802","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.815","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.827","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.838","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.849","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.859","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.868","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.6917105,"monotonic":2345999.654620918,"hcas":{"mlx5_0":{"port_xmit_data":146693084052186,"port_rcv_data":148845502221401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147952460035824,"port_rcv_data":147233289144179,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.789","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.802","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.816","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.828","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.840","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:53.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:53.862","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.872","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.692518,"monotonic":2346000.65542835,"hcas":{"mlx5_0":{"port_xmit_data":146693094323898,"port_rcv_data":148846729279654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147953689764826,"port_rcv_data":147233299523628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.792","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.806","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.831","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.843","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.865","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.875","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.6933222,"monotonic":2346001.656232648,"hcas":{"mlx5_0":{"port_xmit_data":146693104539960,"port_rcv_data":148847946916196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147954906724153,"port_rcv_data":147233309810809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.792","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.806","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.831","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.843","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.865","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.875","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.6941278,"monotonic":2346002.657038377,"hcas":{"mlx5_0":{"port_xmit_data":146693114824843,"port_rcv_data":148849175525820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147956135748064,"port_rcv_data":147233320181802,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.774","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.788","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.800","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:56.812","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:56.823","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:56.834","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:56.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:56.854","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.6949546,"monotonic":2346003.65786545,"hcas":{"mlx5_0":{"port_xmit_data":146693125102803,"port_rcv_data":148850403503815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147957363760874,"port_rcv_data":147233330544293,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.775","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.788","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.801","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.813","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.825","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.845","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.855","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.695783,"monotonic":2346004.658693509,"hcas":{"mlx5_0":{"port_xmit_data":146693135329257,"port_rcv_data":148851621848648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147958582818096,"port_rcv_data":147233340848802,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.792","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.806","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.819","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.831","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.843","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:58.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.864","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.874","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.95,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.6965916,"monotonic":2346005.659502116,"hcas":{"mlx5_0":{"port_xmit_data":146693145624625,"port_rcv_data":148852852223236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147959813140305,"port_rcv_data":147233351233466,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.777","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.791","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.804","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.815","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.826","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.837","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.847","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.856","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.6974177,"monotonic":2346006.6603284,"hcas":{"mlx5_0":{"port_xmit_data":146693155921349,"port_rcv_data":148854081421493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147961039879528,"port_rcv_data":147233361585808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.798","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.811","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.825","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:00.837","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:00.849","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:00.860","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:00.871","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:00.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.69822,"monotonic":2346007.661130528,"hcas":{"mlx5_0":{"port_xmit_data":146693166156188,"port_rcv_data":148855300726393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147962260788670,"port_rcv_data":147233371923917,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.778","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.792","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.805","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.817","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.829","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.839","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.860","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.7,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.6990464,"monotonic":2346008.661956981,"hcas":{"mlx5_0":{"port_xmit_data":146693176502754,"port_rcv_data":148856508350464,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321389263,"port_rcv_data":2420247451055,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147963469686667,"port_rcv_data":147233382363987,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265537621,"port_rcv_data":2294868234246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.818","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.831","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.856","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:02.867","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:02.878","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.898","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.6998322,"monotonic":2346009.662742914,"hcas":{"mlx5_0":{"port_xmit_data":146693187527966,"port_rcv_data":148857646379197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321398982,"port_rcv_data":2420257024175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147964595528975,"port_rcv_data":147233402974592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265547277,"port_rcv_data":2294877807366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.797","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.811","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.824","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.836","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.847","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.858","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.868","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.878","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.70064,"monotonic":2346010.663550366,"hcas":{"mlx5_0":{"port_xmit_data":146693198736419,"port_rcv_data":148858771255396,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321427052,"port_rcv_data":2420285743535,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147965692298898,"port_rcv_data":147233442931823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265575425,"port_rcv_data":2294906526726,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.782","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.796","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.809","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.822","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.834","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.844","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.854","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.865","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.7014623,"monotonic":2346011.664372821,"hcas":{"mlx5_0":{"port_xmit_data":146693210089542,"port_rcv_data":148859897745106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321441064,"port_rcv_data":2420300103215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147966803968737,"port_rcv_data":147233468661648,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265589437,"port_rcv_data":2294920886406,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.821","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.848","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.861","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.872","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.904","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.45,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.702246,"monotonic":2346012.665156273,"hcas":{"mlx5_0":{"port_xmit_data":146693221328486,"port_rcv_data":148861012988602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321455107,"port_rcv_data":2420314462895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147967907006839,"port_rcv_data":147233494325051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265603496,"port_rcv_data":2294935246086,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.783","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.796","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.810","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.822","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:06.833","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:06.844","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.854","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.864","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.7030694,"monotonic":2346013.665979801,"hcas":{"mlx5_0":{"port_xmit_data":146693232586756,"port_rcv_data":148862128992702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321469119,"port_rcv_data":2420328822575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147969008789228,"port_rcv_data":147233519969963,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265617508,"port_rcv_data":2294949605766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.801","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.827","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.840","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.851","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.882","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.7038803,"monotonic":2346014.666790711,"hcas":{"mlx5_0":{"port_xmit_data":146693243785854,"port_rcv_data":148863239844977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321483162,"port_rcv_data":2420343182255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147970105490726,"port_rcv_data":147233545563769,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672265631551,"port_rcv_data":2294963965446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.801","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.815","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.827","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.839","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.850","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.861","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.871","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.77,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.7046907,"monotonic":2346015.667601239,"hcas":{"mlx5_0":{"port_xmit_data":146693254832150,"port_rcv_data":148864327376751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685321483162,"port_rcv_data":2420343182255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147971193249596,"port_rcv_data":147233592024096,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672266336010,"port_rcv_data":2294999331846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.842","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.856","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.869","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.880","gpu_util_pct":99.0,"memory_used_mib":79963.0,"power_w":144.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.892","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.902","gpu_util_pct":99.0,"memory_used_mib":79959.0,"power_w":146.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.912","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":146.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.922","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259210.7054605,"monotonic":2346016.668371006,"hcas":{"mlx5_0":{"port_xmit_data":146693267990721,"port_rcv_data":148865314335256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685322176880,"port_rcv_data":2420443511864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147972078983287,"port_rcv_data":147233669346900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672266405001,"port_rcv_data":2295064320671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.787","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.801","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.814","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.825","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.837","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.847","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.857","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.867","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.7062857,"monotonic":2346017.669196508,"hcas":{"mlx5_0":{"port_xmit_data":146693280414834,"port_rcv_data":148866231815283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685322207152,"port_rcv_data":2420472910740,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147972967988081,"port_rcv_data":147233711177163,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672266435148,"port_rcv_data":2295093777290,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.787","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.801","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.814","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.826","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.837","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:11.848","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.858","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.867","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.7071135,"monotonic":2346018.670024137,"hcas":{"mlx5_0":{"port_xmit_data":146693292812154,"port_rcv_data":148867170743719,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685322265447,"port_rcv_data":2420529663595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147973852476069,"port_rcv_data":147233782428335,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672266495350,"port_rcv_data":2295152603912,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.805","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.819","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.833","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.845","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.856","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.877","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.888","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.707918,"monotonic":2346019.670828365,"hcas":{"mlx5_0":{"port_xmit_data":146693305207614,"port_rcv_data":148868075348684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685322282280,"port_rcv_data":2420546154785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147974736920240,"port_rcv_data":147233809154891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672266509517,"port_rcv_data":2295166963592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.789","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.802","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.815","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.827","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.838","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:13.849","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.859","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.868","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.7087433,"monotonic":2346020.671653974,"hcas":{"mlx5_0":{"port_xmit_data":146693317662976,"port_rcv_data":148868979133556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685322448843,"port_rcv_data":2420569416972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147975620043431,"port_rcv_data":147233871537610,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672267130695,"port_rcv_data":2295216933078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.809","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.822","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.836","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.848","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.859","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.870","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.880","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.890","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.7095501,"monotonic":2346021.672460507,"hcas":{"mlx5_0":{"port_xmit_data":146693330575561,"port_rcv_data":148869908513817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685322959103,"port_rcv_data":2420610483345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147976506419459,"port_rcv_data":147233898282563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672267144847,"port_rcv_data":2295231293452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.790","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.803","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:15.817","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:15.828","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:15.840","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.862","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.871","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.7103763,"monotonic":2346022.673286668,"hcas":{"mlx5_0":{"port_xmit_data":146693342222769,"port_rcv_data":148870866818354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685322968496,"port_rcv_data":2420620056465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147977455558759,"port_rcv_data":147233937217680,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672267483429,"port_rcv_data":2295258549772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.808","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.822","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.835","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.847","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.859","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.869","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.880","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.890","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.711183,"monotonic":2346023.674093687,"hcas":{"mlx5_0":{"port_xmit_data":146693354294899,"port_rcv_data":148871898328249,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685323461257,"port_rcv_data":2420671030859,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147978438890241,"port_rcv_data":147233990937874,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672267833961,"port_rcv_data":2295300682566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.810","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.823","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.836","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.48,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.848","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.860","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.870","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.880","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.890","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.7119915,"monotonic":2346024.674902211,"hcas":{"mlx5_0":{"port_xmit_data":146693365993977,"port_rcv_data":148872954222530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685323708490,"port_rcv_data":2420761071136,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147979407680609,"port_rcv_data":147234084140422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672267917165,"port_rcv_data":2295382366402,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.813","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.852","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.863","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.883","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.893","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.7125502,"monotonic":2346025.675460618,"hcas":{"mlx5_0":{"port_xmit_data":146693377602076,"port_rcv_data":148873978817108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685323754613,"port_rcv_data":2420804903913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147980383476913,"port_rcv_data":147234139035637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672267962131,"port_rcv_data":2295425713464,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.830","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.843","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.857","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.869","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.880","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.891","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.901","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.910","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.09,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.7127697,"monotonic":2346026.675680317,"hcas":{"mlx5_0":{"port_xmit_data":146693389309776,"port_rcv_data":148874993676629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685323798571,"port_rcv_data":2420848251999,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147981356740959,"port_rcv_data":147234194101106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672268005453,"port_rcv_data":2295469060250,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.813","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.851","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:20.862","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.883","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.893","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.71311,"monotonic":2346027.676020398,"hcas":{"mlx5_0":{"port_xmit_data":146693401794908,"port_rcv_data":148875927586992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685323842622,"port_rcv_data":2420891582499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147982245761631,"port_rcv_data":147234249878246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672268049396,"port_rcv_data":2295512389578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.813","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.826","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.838","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.08,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.850","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.860","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.870","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.881","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.890","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.7139206,"monotonic":2346028.676831022,"hcas":{"mlx5_0":{"port_xmit_data":146693414234968,"port_rcv_data":148876845489144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685323872568,"port_rcv_data":2420920569605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147983135145647,"port_rcv_data":147234308964248,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672268446351,"port_rcv_data":2295559061184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.813","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.826","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.839","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.851","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:22.863","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.874","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.884","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.895","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":134.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.714727,"monotonic":2346029.677637193,"hcas":{"mlx5_0":{"port_xmit_data":146693427262837,"port_rcv_data":148877791753395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685324411271,"port_rcv_data":2420971859197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147984032493117,"port_rcv_data":147234363864223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672268813639,"port_rcv_data":2295601510348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.813","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.852","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.863","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.884","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.893","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.7155387,"monotonic":2346030.678449045,"hcas":{"mlx5_0":{"port_xmit_data":146693439842816,"port_rcv_data":148878716243354,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685324620304,"port_rcv_data":2421010630152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147984919284768,"port_rcv_data":147234439508506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672269426803,"port_rcv_data":2295664755334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.813","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.826","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.839","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.851","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.862","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.873","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.884","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.893","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":132.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.7163482,"monotonic":2346031.679258476,"hcas":{"mlx5_0":{"port_xmit_data":146693453093493,"port_rcv_data":148879790746490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685325393540,"port_rcv_data":2421191535646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147985810837098,"port_rcv_data":147234599501190,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672269634224,"port_rcv_data":2295812462887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.796","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.810","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.823","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:25.835","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:25.846","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:25.857","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.867","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.876","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":133.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.717178,"monotonic":2346032.680088576,"hcas":{"mlx5_0":{"port_xmit_data":146693465506150,"port_rcv_data":148880725936795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685325573650,"port_rcv_data":2421225126914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147986715278622,"port_rcv_data":147234663075025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672270127884,"port_rcv_data":2295863737355,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.817","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":140.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.830","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":141.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.856","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.867","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.878","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.898","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.93,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.7179823,"monotonic":2346033.680892598,"hcas":{"mlx5_0":{"port_xmit_data":146693477684426,"port_rcv_data":148881769655237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685326065883,"port_rcv_data":2421280371074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147987703366620,"port_rcv_data":147234712338871,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672270308071,"port_rcv_data":2295901298315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.816","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:27.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.7,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:27.843","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:27.855","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.867","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.877","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.898","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.7187896,"monotonic":2346034.681700295,"hcas":{"mlx5_0":{"port_xmit_data":146693489400573,"port_rcv_data":148882770583386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685326233082,"port_rcv_data":2421303633192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147988682031743,"port_rcv_data":147234765065721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672270779225,"port_rcv_data":2295942365407,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.818","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.831","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.857","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.868","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.879","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.889","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.899","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.7195952,"monotonic":2346035.68250582,"hcas":{"mlx5_0":{"port_xmit_data":146693502507485,"port_rcv_data":148883780962334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685326702964,"port_rcv_data":2421344699634,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147989650739374,"port_rcv_data":147234800978311,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672270946284,"port_rcv_data":2295965628175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.800","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.814","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.84,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.827","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.838","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.849","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.861","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.871","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.7204216,"monotonic":2346036.683332173,"hcas":{"mlx5_0":{"port_xmit_data":146693515820853,"port_rcv_data":148884763497135,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685326868303,"port_rcv_data":2421368039134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147990610801771,"port_rcv_data":147234855454247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672271413624,"port_rcv_data":2296006927413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.800","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":147.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.814","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":146.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.827","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":155.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.839","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.850","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.861","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.871","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":146.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.881","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.7211092,"monotonic":2346037.684019906,"hcas":{"mlx5_0":{"port_xmit_data":146693528780491,"port_rcv_data":148885857234039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685327398976,"port_rcv_data":2421409337722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147991663802755,"port_rcv_data":147234891276243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672271581272,"port_rcv_data":2296030267563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.818","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.832","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.845","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.857","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.869","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.880","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.890","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.901","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.31,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.7219176,"monotonic":2346038.684828187,"hcas":{"mlx5_0":{"port_xmit_data":146693541158277,"port_rcv_data":148886902880112,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685327398976,"port_rcv_data":2421409337722,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147992710562042,"port_rcv_data":147234930224586,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672272102475,"port_rcv_data":2296056792363,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.802","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.816","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.829","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.841","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.852","gpu_util_pct":99.0,"memory_used_mib":79915.0,"power_w":141.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.873","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.882","gpu_util_pct":98.0,"memory_used_mib":80045.0,"power_w":141.99,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.7227416,"monotonic":2346039.685651916,"hcas":{"mlx5_0":{"port_xmit_data":146693553814292,"port_rcv_data":148888140427222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685327971081,"port_rcv_data":2421534841206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147993822323875,"port_rcv_data":147235050041734,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672272363526,"port_rcv_data":2296164911188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.837","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.851","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.864","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:33.876","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.888","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.899","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.909","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.919","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.64,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.7235315,"monotonic":2346040.686441986,"hcas":{"mlx5_0":{"port_xmit_data":146693566328867,"port_rcv_data":148889203240577,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685328136699,"port_rcv_data":2421553255926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147994864688862,"port_rcv_data":147235071982431,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672272372950,"port_rcv_data":2296174484308,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.804","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.831","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.67,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.843","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.855","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.865","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.875","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.885","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.35,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.7243543,"monotonic":2346041.687264763,"hcas":{"mlx5_0":{"port_xmit_data":146693578800115,"port_rcv_data":148890284733273,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685328165234,"port_rcv_data":2421581975493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147995918026196,"port_rcv_data":147235142204891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672272874143,"port_rcv_data":2296232150274,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.805","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.818","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:35.832","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.03,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:35.843","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.01,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.855","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.865","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.876","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.885","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.22,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.7251785,"monotonic":2346042.688088786,"hcas":{"mlx5_0":{"port_xmit_data":146693591750837,"port_rcv_data":148891384061458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685328695055,"port_rcv_data":2421628509275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147996971005006,"port_rcv_data":147235175172397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273024322,"port_rcv_data":2296252587792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.821","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.835","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.848","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.860","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.871","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.882","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.893","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.903","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.725985,"monotonic":2346043.688895684,"hcas":{"mlx5_0":{"port_xmit_data":146693604206274,"port_rcv_data":148892440879613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685328805400,"port_rcv_data":2421645718022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147998012639553,"port_rcv_data":147235201941430,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273038706,"port_rcv_data":2296266947704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.845","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:37.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:37.884","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.895","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.905","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.915","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.925","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.65,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.726771,"monotonic":2346044.689681814,"hcas":{"mlx5_0":{"port_xmit_data":146693616763061,"port_rcv_data":148893515799168,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685328819396,"port_rcv_data":2421660077702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":147999073111920,"port_rcv_data":147235228893066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273052687,"port_rcv_data":2296281307384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.825","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.839","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.852","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":157.96,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.864","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.875","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.897","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.906","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.7275755,"monotonic":2346045.690486173,"hcas":{"mlx5_0":{"port_xmit_data":146693629285922,"port_rcv_data":148894596104244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685328842817,"port_rcv_data":2421684010502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148000127688750,"port_rcv_data":147235265340391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273076108,"port_rcv_data":2296305240184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.842","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.855","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.869","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.881","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.892","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.903","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.913","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.923","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.6,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.728364,"monotonic":2346046.691274542,"hcas":{"mlx5_0":{"port_xmit_data":146693641678899,"port_rcv_data":148895648311465,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685328842817,"port_rcv_data":2421684010502,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148001182582430,"port_rcv_data":147235304344358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273615570,"port_rcv_data":2296331764984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.854","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.881","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:40.893","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:40.905","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.916","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.926","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.936","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.72914,"monotonic":2346047.69205041,"hcas":{"mlx5_0":{"port_xmit_data":146693654723540,"port_rcv_data":148896756300642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685329368902,"port_rcv_data":2421735359022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148002238994365,"port_rcv_data":147235363836534,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273823967,"port_rcv_data":2296380305563,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.826","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.839","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.853","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.865","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.36,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.876","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.887","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.897","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.908","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.7299473,"monotonic":2346048.692857825,"hcas":{"mlx5_0":{"port_xmit_data":146693667383415,"port_rcv_data":148897876823191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685329586476,"port_rcv_data":2421803072184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148003292217441,"port_rcv_data":147235421877973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273868979,"port_rcv_data":2296424304210,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.845","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":159.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.884","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":150.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.894","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.905","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.915","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.924","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.15,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.730739,"monotonic":2346049.693649727,"hcas":{"mlx5_0":{"port_xmit_data":146693679818729,"port_rcv_data":148898956149959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685329616577,"port_rcv_data":2421832608860,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148004342738153,"port_rcv_data":147235463876539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273899530,"port_rcv_data":2296453839926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.830","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.844","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.857","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":159.4,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.870","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.881","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.892","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.902","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.912","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.731543,"monotonic":2346050.694453617,"hcas":{"mlx5_0":{"port_xmit_data":146693692267926,"port_rcv_data":148900034171664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685329647375,"port_rcv_data":2421862145088,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148005388842843,"port_rcv_data":147235505841980,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273929817,"port_rcv_data":2296483376154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.823","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.837","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.850","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.85,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.861","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.873","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.884","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.894","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.904","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.25,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.7323558,"monotonic":2346051.695266535,"hcas":{"mlx5_0":{"port_xmit_data":146693704749718,"port_rcv_data":148901116490458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685329677693,"port_rcv_data":2421891688718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148006441719624,"port_rcv_data":147235547876141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672273960088,"port_rcv_data":2296512919784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.813","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.827","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.840","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.852","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.863","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.874","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.884","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.894","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.11,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.7331793,"monotonic":2346052.696089947,"hcas":{"mlx5_0":{"port_xmit_data":146693717223543,"port_rcv_data":148902194026808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685329703516,"port_rcv_data":2421916438386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148007495120905,"port_rcv_data":147235611645415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672274515469,"port_rcv_data":2296564194252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.833","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.847","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:46.860","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.06,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.873","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.884","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.895","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.905","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.915","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.62,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.7339811,"monotonic":2346053.696891756,"hcas":{"mlx5_0":{"port_xmit_data":146693729973326,"port_rcv_data":148903283732411,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685330052266,"port_rcv_data":2421958753418,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148008544515078,"port_rcv_data":147235657452524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672274701608,"port_rcv_data":2296597668856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.832","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.846","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.860","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.872","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.884","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.894","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.905","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.915","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.7347848,"monotonic":2346054.697695272,"hcas":{"mlx5_0":{"port_xmit_data":146693742793900,"port_rcv_data":148904411182934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685330432311,"port_rcv_data":2422034611543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148009597011808,"port_rcv_data":147235745231170,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672275058821,"port_rcv_data":2296673058544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.870","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":157.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.908","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":149.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.929","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.949","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.7355566,"monotonic":2346055.698467135,"hcas":{"mlx5_0":{"port_xmit_data":146693755442400,"port_rcv_data":148905503724050,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685330618636,"port_rcv_data":2422072794970,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148010651064341,"port_rcv_data":147235805734302,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672275420870,"port_rcv_data":2296721016020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.817","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.830","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.856","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":149.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.867","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.878","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.898","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.7363791,"monotonic":2346056.699289719,"hcas":{"mlx5_0":{"port_xmit_data":146693768271961,"port_rcv_data":148906602289216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685331114838,"port_rcv_data":2422128937192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148011692152622,"port_rcv_data":147235847289066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672275451312,"port_rcv_data":2296750220098,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.817","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.831","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.844","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.856","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.868","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.879","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.889","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.899","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.14,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.7371998,"monotonic":2346057.700110444,"hcas":{"mlx5_0":{"port_xmit_data":146693780276219,"port_rcv_data":148907741286729,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685331144954,"port_rcv_data":2422158140575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148012802066137,"port_rcv_data":147235888524332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672275481196,"port_rcv_data":2296779423481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.838","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.852","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.865","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:51.877","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.889","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.900","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.910","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.920","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.59,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.7380013,"monotonic":2346058.700911892,"hcas":{"mlx5_0":{"port_xmit_data":146693792168817,"port_rcv_data":148908920354471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685331161028,"port_rcv_data":2422172984278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148013966743810,"port_rcv_data":147235932981306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672275858389,"port_rcv_data":2296811950384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:52.819","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:52.832","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:52.845","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.857","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":150.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.868","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.878","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.889","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.898","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":147.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.7388256,"monotonic":2346059.701736024,"hcas":{"mlx5_0":{"port_xmit_data":146693804259261,"port_rcv_data":148910196382922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685331432820,"port_rcv_data":2422283895364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148015133771950,"port_rcv_data":147236064558838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672276297442,"port_rcv_data":2296931736581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.854","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.868","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.881","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.893","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":149.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.905","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.916","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.926","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.936","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.7396128,"monotonic":2346060.702523542,"hcas":{"mlx5_0":{"port_xmit_data":146693816778166,"port_rcv_data":148911377133094,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685331908515,"port_rcv_data":2422310420164,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148016288029053,"port_rcv_data":147236076658417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672276297442,"port_rcv_data":2296931736581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.855","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.869","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.882","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.894","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.905","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.915","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.925","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.935","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.12,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.7403984,"monotonic":2346061.703308779,"hcas":{"mlx5_0":{"port_xmit_data":146693829388951,"port_rcv_data":148912448274817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685332070273,"port_rcv_data":2422328749266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148017340025455,"port_rcv_data":147236125068889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672276768192,"port_rcv_data":2296967577745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.822","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.836","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.849","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.862","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.873","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.885","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.895","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.905","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.88,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.7412176,"monotonic":2346062.704128243,"hcas":{"mlx5_0":{"port_xmit_data":146693842065884,"port_rcv_data":148913523658614,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685332393076,"port_rcv_data":2422360620565,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148018386092883,"port_rcv_data":147236164242073,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672276940242,"port_rcv_data":2296994283746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.843","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.857","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:56.870","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.881","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.893","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.904","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.914","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.924","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.47,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.7420187,"monotonic":2346063.704929307,"hcas":{"mlx5_0":{"port_xmit_data":146693854507296,"port_rcv_data":148914660635352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685332582021,"port_rcv_data":2422398095838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148019485650491,"port_rcv_data":147236228260526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672277418479,"port_rcv_data":2297046119076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.862","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.875","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.889","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.901","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.77,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.912","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.933","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.943","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.91,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.742803,"monotonic":2346064.705713793,"hcas":{"mlx5_0":{"port_xmit_data":146693866861352,"port_rcv_data":148915868655830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685333014409,"port_rcv_data":2422437946590,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148020652161769,"port_rcv_data":147236264170301,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672277600883,"port_rcv_data":2297069968958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.842","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.855","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.868","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":159.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.880","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":150.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.891","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.902","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.912","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.921","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.7436113,"monotonic":2346065.706521736,"hcas":{"mlx5_0":{"port_xmit_data":146693879004489,"port_rcv_data":148917060345528,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685333234354,"port_rcv_data":2422463010084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148021821104847,"port_rcv_data":147236290524297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672277614957,"port_rcv_data":2297084329101,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.841","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.854","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.867","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.880","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.892","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.903","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.913","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.923","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.71,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.7444215,"monotonic":2346066.707331771,"hcas":{"mlx5_0":{"port_xmit_data":146693891044652,"port_rcv_data":148918252553179,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685333248335,"port_rcv_data":2422477369764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148022996768449,"port_rcv_data":147236316926209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672277629000,"port_rcv_data":2297098688781,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:00.862","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:00.876","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.900","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.911","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.932","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.942","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.7452083,"monotonic":2346067.708118915,"hcas":{"mlx5_0":{"port_xmit_data":146693902989278,"port_rcv_data":148919427266560,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685333248335,"port_rcv_data":2422477369764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148024174803444,"port_rcv_data":147236355473766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672278158976,"port_rcv_data":2297125213581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.845","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.884","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.64,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.896","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.907","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.917","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.13,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.927","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.7460124,"monotonic":2346068.708922896,"hcas":{"mlx5_0":{"port_xmit_data":146693915626415,"port_rcv_data":148920732198226,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685333829539,"port_rcv_data":2422604488854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148025353149116,"port_rcv_data":147236476912481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672278442440,"port_rcv_data":2297234675087,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.862","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.876","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.889","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.901","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.912","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.923","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.933","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.942","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.37,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.7468007,"monotonic":2346069.709711048,"hcas":{"mlx5_0":{"port_xmit_data":146693927722282,"port_rcv_data":148921910712766,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685333985515,"port_rcv_data":2422613330592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148026520946525,"port_rcv_data":147236515424185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672278892421,"port_rcv_data":2297261175018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.845","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.859","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.872","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":158.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.884","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.896","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.907","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.918","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.928","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.75,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.747605,"monotonic":2346070.710515404,"hcas":{"mlx5_0":{"port_xmit_data":146693940216142,"port_rcv_data":148923117456636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334440549,"port_rcv_data":2422639830336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148027702278784,"port_rcv_data":147236536633951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045902,"port_rcv_data":2297270285124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.844","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.857","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.871","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":157.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.882","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.894","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.904","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.915","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.925","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.52,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.748413,"monotonic":2346071.711323734,"hcas":{"mlx5_0":{"port_xmit_data":146693952134075,"port_rcv_data":148924300203708,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148028876034915,"port_rcv_data":147236548446855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.866","gpu_util_pct":97.0,"memory_used_mib":79997.0,"power_w":150.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.879","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:05.892","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.904","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.916","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.927","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.937","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.947","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.24,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.7492003,"monotonic":2346072.712111067,"hcas":{"mlx5_0":{"port_xmit_data":146693963457850,"port_rcv_data":148925529253046,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148030106113548,"port_rcv_data":147236559845577,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.865","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.879","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.892","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":158.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.904","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.916","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.937","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.947","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.5,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.7499878,"monotonic":2346073.712898324,"hcas":{"mlx5_0":{"port_xmit_data":146693974752051,"port_rcv_data":148926755615108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148031332575314,"port_rcv_data":147236571211926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.848","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.06,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.875","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.888","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.899","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.910","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.920","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.930","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":141.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.7507946,"monotonic":2346074.713705513,"hcas":{"mlx5_0":{"port_xmit_data":146693986095097,"port_rcv_data":148927986786857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148032563722664,"port_rcv_data":147236582622004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:08.849","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:08.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.875","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.887","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.898","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.909","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.919","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.928","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.94,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.7516022,"monotonic":2346075.714512683,"hcas":{"mlx5_0":{"port_xmit_data":146693998086752,"port_rcv_data":148929168055585,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148033742748984,"port_rcv_data":147236594638009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.849","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.862","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.876","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":156.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.888","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.0,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.899","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":142.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.910","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.920","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.930","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.7524085,"monotonic":2346076.715319105,"hcas":{"mlx5_0":{"port_xmit_data":146694009964694,"port_rcv_data":148930332282004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148034908399908,"port_rcv_data":147236606583524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.853","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.867","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.879","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.891","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.902","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.913","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.923","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.933","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.0,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.7532158,"monotonic":2346077.716126225,"hcas":{"mlx5_0":{"port_xmit_data":146694021887556,"port_rcv_data":148931498892525,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148036075466127,"port_rcv_data":147236618569019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.853","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:11.866","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.879","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":158.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.891","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.902","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.912","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.923","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.932","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.9,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.754023,"monotonic":2346078.716933865,"hcas":{"mlx5_0":{"port_xmit_data":146694033882021,"port_rcv_data":148932674301957,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148037251621411,"port_rcv_data":147236630614846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:12.870","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.98,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.908","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.920","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.931","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.941","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.951","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":142.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.7548137,"monotonic":2346079.717724124,"hcas":{"mlx5_0":{"port_xmit_data":146694045817949,"port_rcv_data":148933844146270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148038421095473,"port_rcv_data":147236642602085,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.857","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.870","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.883","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.895","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.906","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.917","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.927","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.937","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.26,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.7556186,"monotonic":2346080.718529071,"hcas":{"mlx5_0":{"port_xmit_data":146694057821391,"port_rcv_data":148935021163442,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148039598302880,"port_rcv_data":147236654649325,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.853","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.866","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.879","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.891","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":149.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.902","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":144.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.913","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.923","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.932","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.66,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.7564278,"monotonic":2346081.719338284,"hcas":{"mlx5_0":{"port_xmit_data":146694069717953,"port_rcv_data":148936186753291,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148040764432170,"port_rcv_data":147236666596826,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.857","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.870","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.884","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":155.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.896","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":151.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.907","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":145.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.917","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.928","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.938","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":143.74,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.7572343,"monotonic":2346082.72014507,"hcas":{"mlx5_0":{"port_xmit_data":146694081711677,"port_rcv_data":148937363544458,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148041940704472,"port_rcv_data":147236678626527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:16.857","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:16.871","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:16.884","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":159.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.895","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":150.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.906","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.917","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.927","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.937","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":146.28,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.7580426,"monotonic":2346083.72095329,"hcas":{"mlx5_0":{"port_xmit_data":146694093573694,"port_rcv_data":148938526060888,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148043103046689,"port_rcv_data":147236690554288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.875","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.888","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.901","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":159.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.914","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":147.2,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.926","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":146.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.937","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.947","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.956","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":145.21,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.7588305,"monotonic":2346084.721741218,"hcas":{"mlx5_0":{"port_xmit_data":146694105633028,"port_rcv_data":148939659272447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148044236360912,"port_rcv_data":147236702662164,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:18.857","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:18.870","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.883","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":159.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.895","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":148.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.905","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.916","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.926","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.935","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.96,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.75964,"monotonic":2346085.722550586,"hcas":{"mlx5_0":{"port_xmit_data":146694117965124,"port_rcv_data":148940701559701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148045276876260,"port_rcv_data":147236715012921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.857","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.872","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.887","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.899","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.911","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":143.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.933","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.943","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":144.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.760441,"monotonic":2346086.723351515,"hcas":{"mlx5_0":{"port_xmit_data":146694131197889,"port_rcv_data":148941661071292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148046238450727,"port_rcv_data":147236728297851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.878","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.892","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.905","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.917","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.928","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.938","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.948","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.957","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.51,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.7610826,"monotonic":2346087.723993019,"hcas":{"mlx5_0":{"port_xmit_data":146694144662280,"port_rcv_data":148942627196034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148047205219361,"port_rcv_data":147236741806060,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.861","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.875","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.900","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.911","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.921","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.931","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.33,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.941","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.57,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.7618904,"monotonic":2346088.724800844,"hcas":{"mlx5_0":{"port_xmit_data":146694158056304,"port_rcv_data":148943586084026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148048161531997,"port_rcv_data":147236755172367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:22.861","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:22.875","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.888","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":153.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.900","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.911","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.922","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.932","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.941","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":137.76,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.7626972,"monotonic":2346089.725607818,"hcas":{"mlx5_0":{"port_xmit_data":146694171404829,"port_rcv_data":148944542247081,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148049120635346,"port_rcv_data":147236768588993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.878","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.892","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.905","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.917","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.928","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.939","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.17,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.949","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.959","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":140.86,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.763493,"monotonic":2346090.72640358,"hcas":{"mlx5_0":{"port_xmit_data":146694184713508,"port_rcv_data":148945495468825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148050073177942,"port_rcv_data":147236781913180,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.845","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.858","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.871","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.883","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.895","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":140.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.906","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.917","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.926","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":139.82,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.7643192,"monotonic":2346091.727229623,"hcas":{"mlx5_0":{"port_xmit_data":146694198084422,"port_rcv_data":148946453086433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148051028666754,"port_rcv_data":147236795267006,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.884","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.897","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.908","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.916","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.925","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.934","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.943","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.951","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.97,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.7651227,"monotonic":2346092.728033114,"hcas":{"mlx5_0":{"port_xmit_data":146694211467782,"port_rcv_data":148947412232118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148051989974201,"port_rcv_data":147236808712588,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.869","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":154.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.908","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":138.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.929","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.949","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.81,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.7659233,"monotonic":2346093.728833742,"hcas":{"mlx5_0":{"port_xmit_data":146694224872567,"port_rcv_data":148948372202466,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148052950137898,"port_rcv_data":147236822129794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:27.882","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.896","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.909","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.922","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.933","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":141.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.944","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.954","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.964","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":138.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.7667112,"monotonic":2346094.729621761,"hcas":{"mlx5_0":{"port_xmit_data":146694238895314,"port_rcv_data":148949202430544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148053778136901,"port_rcv_data":147236836153105,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.882","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.05,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.895","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.909","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.06,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.920","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.931","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":137.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.942","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.952","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.962","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.23,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.7675006,"monotonic":2346095.730410951,"hcas":{"mlx5_0":{"port_xmit_data":146694253206850,"port_rcv_data":148950007382314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148054583126454,"port_rcv_data":147236850482620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.865","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.78,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.879","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.892","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.904","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.916","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":135.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.926","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.936","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.946","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.72,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.7683067,"monotonic":2346096.731217089,"hcas":{"mlx5_0":{"port_xmit_data":146694267324284,"port_rcv_data":148950803956760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148055381251534,"port_rcv_data":147236864647230,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:30.869","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.26,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:30.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.31,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.908","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":139.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.940","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.950","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.7691097,"monotonic":2346097.732020393,"hcas":{"mlx5_0":{"port_xmit_data":146694281691607,"port_rcv_data":148951609279433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148056185101090,"port_rcv_data":147236879005710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.868","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.882","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.895","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.907","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":136.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.929","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.96,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.939","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.949","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":135.43,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.7699137,"monotonic":2346098.732824033,"hcas":{"mlx5_0":{"port_xmit_data":146694295950974,"port_rcv_data":148952411503566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148056989077875,"port_rcv_data":147236893331721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.869","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.92,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":152.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.908","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.940","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.950","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":136.39,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.7707202,"monotonic":2346099.733630841,"hcas":{"mlx5_0":{"port_xmit_data":146694309642556,"port_rcv_data":148953077862730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148057654134940,"port_rcv_data":147236907018032,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.869","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.908","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":134.41,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.940","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.950","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.49,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.7715266,"monotonic":2346100.734437178,"hcas":{"mlx5_0":{"port_xmit_data":146694323023764,"port_rcv_data":148953667434387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148058243373842,"port_rcv_data":147236920406111,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.869","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.93,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.883","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.896","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.908","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.919","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":129.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.930","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.33,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.940","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.949","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":129.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.7723389,"monotonic":2346101.735249354,"hcas":{"mlx5_0":{"port_xmit_data":146694336388409,"port_rcv_data":148954256636510,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148058833962393,"port_rcv_data":147236933823683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.873","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.899","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.52,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.910","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.921","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":130.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.931","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.941","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.951","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":131.63,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.7731504,"monotonic":2346102.736060807,"hcas":{"mlx5_0":{"port_xmit_data":146694349810756,"port_rcv_data":148954848576721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148059426302840,"port_rcv_data":147236947267713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.873","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.886","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.900","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.912","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.923","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":133.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.933","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.944","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.953","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":130.4,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.7739582,"monotonic":2346103.736868577,"hcas":{"mlx5_0":{"port_xmit_data":146694359572359,"port_rcv_data":148955289650454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148059866362202,"port_rcv_data":147236957017638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.890","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.903","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.915","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.28,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.927","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.937","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":127.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.947","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.29,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.957","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.966","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":125.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.7747545,"monotonic":2346104.737664951,"hcas":{"mlx5_0":{"port_xmit_data":146694366569017,"port_rcv_data":148955613579710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148060189620295,"port_rcv_data":147236963999484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.909","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":129.84,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.923","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":126.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.935","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.948","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":125.46,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.958","gpu_util_pct":100.0,"memory_used_mib":79915.0,"power_w":125.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.969","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.979","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.989","gpu_util_pct":100.0,"memory_used_mib":80045.0,"power_w":123.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.7755282,"monotonic":2346105.738438653,"hcas":{"mlx5_0":{"port_xmit_data":146694368440430,"port_rcv_data":148955701305670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148060277346205,"port_rcv_data":147236965870056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.893","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":92.92,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.911","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":90.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.928","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":89.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.944","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":92.03,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.959","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":90.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.974","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":86.84,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.988","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":86.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.002","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":90.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.776283,"monotonic":2346106.739193643,"hcas":{"mlx5_0":{"port_xmit_data":146694368440430,"port_rcv_data":148955701305670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148060277346205,"port_rcv_data":147236965870056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.925","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":84.16,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.943","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":84.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.959","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":88.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.974","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":87.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.987","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":85.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.999","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":86.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.010","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":85.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.020","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":81.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.7770286,"monotonic":2346107.739938939,"hcas":{"mlx5_0":{"port_xmit_data":146694368440430,"port_rcv_data":148955701305670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148060277346205,"port_rcv_data":147236965870056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.900","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":83.53,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.918","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.935","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.949","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":83.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.961","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":82.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.974","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.96,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.986","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":80.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.999","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":81.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259302.7777863,"monotonic":2346108.740696588,"hcas":{"mlx5_0":{"port_xmit_data":146694368440430,"port_rcv_data":148955701305670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148060277346205,"port_rcv_data":147236965870056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:42.924","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":83.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:42.941","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:42.958","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.37,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:42.973","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":82.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:42.988","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":81.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:43.001","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.91,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:43.012","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.2,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:43.024","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":81.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259303.7785256,"monotonic":2346109.74143594,"hcas":{"mlx5_0":{"port_xmit_data":146694368440430,"port_rcv_data":148955701305670,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":2685334592108,"port_rcv_data":2422648939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":148060277346205,"port_rcv_data":147236965870056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2672279045995,"port_rcv_data":2297270285587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:43.923","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":83.66,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:43.941","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:43.957","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":82.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:43.972","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":82.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:43.986","gpu_util_pct":0.0,"memory_used_mib":79915.0,"power_w":81.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:44.000","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:44.014","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:44.026","gpu_util_pct":0.0,"memory_used_mib":80045.0,"power_w":81.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-ab-pmem084-20260901-1823/telemetry/node_8.jsonl

@@ -0,0 +1,459 @@
+{"wall_time":1788258845.7249591,"monotonic":2345683.174026777,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:05.849","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:05.876","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:05.892","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:05.907","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:05.922","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:05.936","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:05.950","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:05.963","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":78.02,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258846.7257016,"monotonic":2345684.174769593,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:06.846","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:06.864","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:06.881","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.17,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:06.897","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:06.912","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:06.926","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:06.940","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.09,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:06.953","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258847.725995,"monotonic":2345685.175062271,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:07.826","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:07.844","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:07.862","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:07.877","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:07.893","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:07.907","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:07.921","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:07.935","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258848.7267652,"monotonic":2345686.175832622,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:08.850","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:08.869","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:08.887","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:08.903","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:08.919","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:08.933","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:08.947","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:08.961","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258849.7275085,"monotonic":2345687.176576201,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:09.865","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:09.884","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:09.902","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:09.918","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:09.933","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:09.948","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:09.962","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.8,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:09.976","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258850.72824,"monotonic":2345688.177307993,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:10.873","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:10.892","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.59,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:10.909","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:10.926","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:10.941","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:10.955","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:10.969","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:10.983","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258851.728962,"monotonic":2345689.178029807,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:11.826","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:11.844","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:11.861","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.0,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:11.877","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:11.892","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:11.906","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:11.920","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.64,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:11.934","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258852.7297342,"monotonic":2345690.178801955,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:12.845","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:12.864","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:12.881","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:12.898","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:12.914","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:12.929","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:12.945","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.39,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:12.959","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258853.7304814,"monotonic":2345691.179549259,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:13.873","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:13.892","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:13.909","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.14,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:13.924","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:13.939","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:13.954","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:13.968","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:13.982","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258854.7309725,"monotonic":2345692.180040356,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:14.853","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:14.872","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:14.889","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.17,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:14.905","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:14.920","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:14.935","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.7,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:14.949","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:14.963","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258855.7317176,"monotonic":2345693.18078541,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:15.873","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:15.891","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:15.909","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:15.925","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:15.941","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:15.956","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.7,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:15.970","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.09,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:15.984","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258856.7324417,"monotonic":2345694.181509503,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:16.873","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:16.891","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:16.908","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:16.924","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:16.939","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:16.954","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:16.968","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.09,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:16.982","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258857.7331707,"monotonic":2345695.18223847,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:17.832","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:17.851","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:17.868","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:17.884","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:17.900","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:17.914","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:17.928","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:17.942","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258858.733608,"monotonic":2345696.182675458,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:18.854","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:18.872","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:18.889","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:18.904","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:18.919","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:18.933","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:18.947","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:18.960","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258859.7343636,"monotonic":2345697.183431451,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:19.853","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:19.871","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:19.887","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:19.903","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:19.917","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:19.931","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:19.945","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:19.958","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258860.7351062,"monotonic":2345698.184174135,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:20.877","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:20.895","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:20.913","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:20.928","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:20.943","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:20.958","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:20.971","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.64,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:20.985","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258861.7358215,"monotonic":2345699.184889561,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:21.861","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:21.879","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:21.896","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:21.911","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:21.926","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:21.940","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:21.953","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:21.966","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258862.736567,"monotonic":2345700.185634912,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:22.881","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:22.899","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:22.915","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.0,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:22.931","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:22.945","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:22.960","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:22.973","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.09,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:22.986","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258863.7372947,"monotonic":2345701.186362386,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:23.877","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:23.895","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:23.912","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:23.927","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:23.942","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:23.956","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:23.969","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:23.982","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258864.738027,"monotonic":2345702.187095139,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:24.857","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:24.875","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:24.892","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:24.908","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:24.923","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:24.937","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:24.951","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:24.964","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258865.7387822,"monotonic":2345703.187850131,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:25.885","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:25.903","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:25.920","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:25.936","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:25.950","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:25.964","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:25.978","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:25.991","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258866.7395113,"monotonic":2345704.188579287,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:26.841","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.83,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:26.859","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:26.876","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:26.892","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:26.907","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:26.921","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:26.935","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:26.948","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258867.7402737,"monotonic":2345705.189341577,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:27.861","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:27.879","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:27.896","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:27.912","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:27.927","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:27.941","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:27.955","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:27.968","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258868.7410228,"monotonic":2345706.190090635,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:28.881","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:28.899","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:28.916","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.81,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:28.931","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:28.946","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:28.960","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:28.974","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:28.987","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258869.7417462,"monotonic":2345707.190814122,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:29.881","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:29.899","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:29.916","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:29.932","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:29.946","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:29.961","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:29.974","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.85,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:29.987","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258870.7424815,"monotonic":2345708.191549376,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:30.885","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:30.903","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:30.919","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.17,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:30.935","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:30.949","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:30.963","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:30.977","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:30.990","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258871.7432163,"monotonic":2345709.192284399,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:31.886","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:31.904","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:31.922","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.22,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:31.938","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:31.953","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:31.968","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:31.982","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:31.995","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258872.7439454,"monotonic":2345710.193013033,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:32.886","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:32.905","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:32.922","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:32.938","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:32.953","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:32.967","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:32.981","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:32.994","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258873.7446597,"monotonic":2345711.193727601,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:33.886","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:33.904","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:33.921","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:33.937","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:33.952","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:33.966","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:33.980","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.29,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:33.993","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258874.7453754,"monotonic":2345712.19444334,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:34.861","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:34.880","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:34.897","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:34.913","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:34.928","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:34.943","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:34.957","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:34.970","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258875.746116,"monotonic":2345713.195183607,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:35.865","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:35.883","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:35.901","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.92,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:35.916","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:35.931","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:35.945","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:35.959","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.09,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:35.973","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258876.7468672,"monotonic":2345714.195935216,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:36.865","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:36.878","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:36.892","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.92,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:36.905","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:36.919","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:36.932","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:36.945","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:36.957","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258877.7476196,"monotonic":2345715.196687427,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:37.893","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:37.911","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:37.928","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:37.944","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:37.958","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:37.973","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:37.986","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.000","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258878.7483435,"monotonic":2345716.197411565,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:38.889","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:38.907","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:38.925","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.75,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:38.941","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:38.956","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:38.971","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:38.985","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:38.998","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258879.7490578,"monotonic":2345717.198125824,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:39.865","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:39.884","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:39.901","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:39.917","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:39.931","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:39.946","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:39.960","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:39.973","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258880.7498095,"monotonic":2345718.198877291,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:40.889","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:40.907","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:40.924","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.71,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:40.940","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:40.955","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:40.969","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:40.983","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:40.996","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258881.7505267,"monotonic":2345719.199594647,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:41.893","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:41.911","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:41.929","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:41.944","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:41.959","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:41.974","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:41.988","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:42.001","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258882.7512527,"monotonic":2345720.200320672,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:42.897","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:42.916","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:42.933","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:42.950","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:42.965","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:42.980","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:42.994","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:43.008","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258883.7519593,"monotonic":2345721.201027189,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:43.897","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:43.915","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:43.933","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:43.948","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:43.963","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:43.977","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:43.991","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.004","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258884.7526817,"monotonic":2345722.201749697,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:44.853","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:44.871","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:44.888","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:44.903","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:44.918","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:44.932","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:44.946","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:44.959","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258885.753436,"monotonic":2345723.202504106,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:45.873","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:45.891","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:45.909","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.94,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:45.925","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:45.939","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:45.954","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:45.968","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:45.981","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258886.7541888,"monotonic":2345724.203256431,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:46.853","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:46.871","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:46.888","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.12,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:46.905","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.8,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:46.920","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:46.934","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:46.948","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.97,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:46.962","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258887.7549484,"monotonic":2345725.204016342,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:47.897","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.74,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:47.915","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:47.933","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.36,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:47.949","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.82,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:47.964","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:47.979","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:47.993","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.006","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258888.7556713,"monotonic":2345726.2047392,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:48.873","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.47,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:48.892","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:48.909","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.87,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:48.925","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:48.939","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:48.954","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:48.968","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.61,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:48.981","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258889.7564108,"monotonic":2345727.205478835,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:49.856","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:49.875","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:49.892","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:49.908","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:49.923","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:49.938","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:49.952","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:49.966","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258890.7571712,"monotonic":2345728.206239652,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:50.857","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:50.876","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:50.893","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.81,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:50.909","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:50.924","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:50.939","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:50.953","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:50.966","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258891.7579234,"monotonic":2345729.206991247,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:51.905","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:51.923","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:51.941","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.05,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:51.956","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:51.971","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:51.985","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:51.998","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:52.010","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258892.7586532,"monotonic":2345730.207720981,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:52.901","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":76.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:52.919","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:52.936","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:52.952","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:52.967","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:52.982","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:52.995","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.009","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258893.7593725,"monotonic":2345731.208440348,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:53.857","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:53.875","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:53.892","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:53.908","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:53.923","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:53.937","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:53.951","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:53.964","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258894.760147,"monotonic":2345732.209214989,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:54.881","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:54.900","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:54.917","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:54.933","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:54.948","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:54.963","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:54.977","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:54.990","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258895.7608817,"monotonic":2345733.209949612,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:55.838","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:55.856","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:55.873","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:55.889","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:55.903","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:55.918","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:55.932","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:55.945","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258896.7616732,"monotonic":2345734.210741204,"hcas":{"mlx5_0":{"port_xmit_data":82835861523799,"port_rcv_data":84154237889840,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876649129,"port_rcv_data":3193581957025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331332864,"port_rcv_data":82214610976013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392628923,"port_rcv_data":2266312876605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:56.864","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:56.883","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:56.900","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:56.916","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:56.931","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:56.946","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:56.960","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:56.973","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258897.7624242,"monotonic":2345735.21149228,"hcas":{"mlx5_0":{"port_xmit_data":82835861533223,"port_rcv_data":84154247462960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876658553,"port_rcv_data":3193591530145,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331342241,"port_rcv_data":82214620549133,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392638300,"port_rcv_data":2266322449725,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:57.882","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:57.901","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:57.918","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.0,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:57.934","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:57.949","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:57.963","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:57.977","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:57.990","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258898.7631593,"monotonic":2345736.212227335,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:58.865","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:58.884","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:58.901","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.71,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:58.917","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:58.932","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:58.946","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:58.960","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:58.974","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258899.7639115,"monotonic":2345737.212979358,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:34:59.861","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:34:59.880","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:34:59.897","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:34:59.913","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:34:59.928","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:34:59.942","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:34:59.956","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.85,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:34:59.970","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258900.764701,"monotonic":2345738.213768447,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:00.844","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:00.863","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:00.881","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:00.897","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:00.912","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:00.927","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:00.941","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:00.955","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258901.7654915,"monotonic":2345739.214559247,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:01.864","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:01.883","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:01.900","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:01.915","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:01.930","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:01.945","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:01.958","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:01.972","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258902.7662647,"monotonic":2345740.215332194,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:02.885","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:02.903","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:02.921","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:02.937","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:02.952","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:02.966","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:02.980","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:02.994","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258903.767002,"monotonic":2345741.216069812,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:03.869","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:03.887","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:03.904","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:03.920","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:03.935","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:03.949","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:03.963","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:03.976","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":76.56,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258904.7677696,"monotonic":2345742.216837402,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:04.864","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:04.882","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:04.900","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:04.915","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:04.930","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:04.944","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:04.958","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:04.972","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":76.8,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258905.7685432,"monotonic":2345743.217611016,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:05.869","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:05.887","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:05.905","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.71,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:05.921","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:05.936","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:05.950","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:05.964","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:05.977","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258906.7693095,"monotonic":2345744.218377351,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:06.848","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:06.866","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:06.883","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:06.898","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:06.913","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:06.928","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:06.941","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:06.955","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258907.7701025,"monotonic":2345745.21917053,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:07.893","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:07.908","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:07.921","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:07.935","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:07.948","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:07.961","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:07.973","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:07.986","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258908.7708652,"monotonic":2345746.219932987,"hcas":{"mlx5_0":{"port_xmit_data":82835861547204,"port_rcv_data":84154261822640,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876672565,"port_rcv_data":3193605889825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331356145,"port_rcv_data":82214634908813,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392652080,"port_rcv_data":2266336809405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:08.849","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:08.868","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:08.886","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:08.902","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:08.917","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:08.931","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:08.946","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:08.959","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258909.771635,"monotonic":2345747.220702966,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:09.851","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:09.870","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:09.887","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:09.903","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:09.918","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:09.932","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:09.946","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:09.960","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258910.7724364,"monotonic":2345748.221504192,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:10.868","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:10.886","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:10.904","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:10.920","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:10.935","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:10.949","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:10.963","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.17,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:10.976","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258911.7732081,"monotonic":2345749.222275685,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:11.913","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:11.931","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:11.948","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:11.963","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:11.978","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:11.992","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.006","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.019","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258912.7739518,"monotonic":2345750.223019312,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:12.873","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:12.892","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:12.909","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.94,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:12.925","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:12.941","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:12.955","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:12.969","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:12.983","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258913.7747178,"monotonic":2345751.223785581,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:13.853","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:13.872","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:13.889","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:13.905","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:13.920","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:13.935","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:13.949","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:13.962","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258914.7755504,"monotonic":2345752.224618126,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:14.878","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:14.896","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:14.913","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:14.928","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:14.943","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:14.957","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:14.971","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:14.985","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258915.776313,"monotonic":2345753.225380807,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:15.893","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:15.911","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.18,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:15.928","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:15.944","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:15.959","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:15.974","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:15.987","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.85,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:16.001","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258916.7770762,"monotonic":2345754.226143857,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:16.897","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:16.915","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:16.933","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:16.949","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:16.964","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:16.978","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:16.992","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:17.006","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258917.7778132,"monotonic":2345755.226881032,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:17.898","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:17.916","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:17.933","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:17.949","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:17.964","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:17.979","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:17.993","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.66,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.006","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258918.778568,"monotonic":2345756.227635582,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:18.882","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:18.900","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:18.917","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.29,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:18.932","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:18.947","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:18.961","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:18.975","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:18.988","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258919.7793412,"monotonic":2345757.228409175,"hcas":{"mlx5_0":{"port_xmit_data":82835861575228,"port_rcv_data":84154290542000,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876700604,"port_rcv_data":3193634609185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331384107,"port_rcv_data":82214663628173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392680057,"port_rcv_data":2266365528765,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:19.926","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:19.944","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:19.961","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.0,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:19.976","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:19.991","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.005","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.019","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.034","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258920.7800667,"monotonic":2345758.229134619,"hcas":{"mlx5_0":{"port_xmit_data":82835861589240,"port_rcv_data":84154304901680,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876714585,"port_rcv_data":3193648968865,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331398041,"port_rcv_data":82214677987853,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392694023,"port_rcv_data":2266379888445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:20.882","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:20.900","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:20.917","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:20.933","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:20.948","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:20.963","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:20.976","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:20.990","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258921.7808378,"monotonic":2345759.229905454,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:21.918","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.68,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:21.936","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:21.954","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:21.970","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:21.985","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:21.999","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:22.013","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.65,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:22.027","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258922.7815557,"monotonic":2345760.230623595,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:22.929","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:22.947","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:22.965","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:22.980","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:22.995","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.010","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.023","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.037","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258923.782266,"monotonic":2345761.231333658,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:23.877","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:23.895","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:23.913","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.92,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:23.928","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:23.943","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:23.958","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:23.972","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:23.985","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258924.7829587,"monotonic":2345762.232026842,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:24.905","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:24.923","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:24.940","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.92,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:24.956","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:24.971","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:24.986","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.000","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.014","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258925.783705,"monotonic":2345763.232773436,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:25.881","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:25.900","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:25.917","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.14,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:25.933","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:25.948","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:25.963","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:25.977","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:25.990","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258926.7844753,"monotonic":2345764.233543453,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:26.885","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:26.904","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:26.920","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.17,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:26.936","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:26.951","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:26.965","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:26.978","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:26.992","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258927.7852287,"monotonic":2345765.234296939,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:27.905","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:27.923","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:27.940","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:27.956","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:27.970","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:27.985","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:27.998","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.07,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.012","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258928.7859755,"monotonic":2345766.235043484,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:28.864","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:28.883","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:28.900","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:28.915","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:28.930","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:28.945","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:28.958","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.07,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:28.972","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258929.7867527,"monotonic":2345767.235820648,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:29.913","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:29.931","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:29.948","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.02,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:29.964","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:29.979","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:29.993","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.007","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.021","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258930.787481,"monotonic":2345768.236549122,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:30.889","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:30.907","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:30.925","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:30.941","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:30.956","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:30.970","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:30.984","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.8,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:30.998","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258931.7882466,"monotonic":2345769.23731452,"hcas":{"mlx5_0":{"port_xmit_data":82835861603252,"port_rcv_data":84154319261360,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876728597,"port_rcv_data":3193663328545,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331412038,"port_rcv_data":82214692347533,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392707988,"port_rcv_data":2266394248125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:31.884","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:31.902","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:31.920","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.71,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:31.936","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:31.951","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:31.966","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:31.980","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:31.994","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258932.7890143,"monotonic":2345770.238082229,"hcas":{"mlx5_0":{"port_xmit_data":82835861617264,"port_rcv_data":84154333621040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876742609,"port_rcv_data":3193677688225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331425957,"port_rcv_data":82214706707213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555392721985,"port_rcv_data":2266408607805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:32.888","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:32.907","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:32.924","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.8,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:32.939","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:32.954","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:32.968","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:32.982","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.15,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:32.995","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258933.7897713,"monotonic":2345771.238839266,"hcas":{"mlx5_0":{"port_xmit_data":82835861617264,"port_rcv_data":84154333621040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438876742609,"port_rcv_data":3193677688225,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627331937891,"port_rcv_data":82214733232013,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555393232415,"port_rcv_data":2266435132605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:33.889","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:33.907","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:33.925","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.99,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:33.941","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:33.956","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:33.971","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:33.985","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.14,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:33.999","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258934.7905278,"monotonic":2345772.239595453,"hcas":{"mlx5_0":{"port_xmit_data":82835862034532,"port_rcv_data":84154448157603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438877169178,"port_rcv_data":3193792155133,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627332195935,"port_rcv_data":82214838905575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555393489467,"port_rcv_data":2266540777650,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:34.914","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:34.932","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:34.950","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:34.966","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:34.981","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:34.996","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.010","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.024","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258935.7912664,"monotonic":2345773.240334247,"hcas":{"mlx5_0":{"port_xmit_data":82835862352685,"port_rcv_data":84154465840922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438877483580,"port_rcv_data":3193809838548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627332500215,"port_rcv_data":82214855837230,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555393790183,"port_rcv_data":2266557692577,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:35.870","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:35.889","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:35.905","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":76.93,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:35.921","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:35.936","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:35.950","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:35.964","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:35.977","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258936.7920601,"monotonic":2345774.241128095,"hcas":{"mlx5_0":{"port_xmit_data":82835862504616,"port_rcv_data":84154474410646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438877633574,"port_rcv_data":3193818400348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627332801411,"port_rcv_data":82214873183464,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394090077,"port_rcv_data":2266575021747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:36.917","gpu_util_pct":0.0,"memory_used_mib":79971.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:36.936","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:36.953","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:36.969","gpu_util_pct":0.0,"memory_used_mib":79937.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:36.985","gpu_util_pct":0.0,"memory_used_mib":79849.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:36.999","gpu_util_pct":0.0,"memory_used_mib":79933.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:37.013","gpu_util_pct":0.0,"memory_used_mib":79917.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:37.027","gpu_util_pct":0.0,"memory_used_mib":79979.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258937.7927918,"monotonic":2345775.241859119,"hcas":{"mlx5_0":{"port_xmit_data":82835864725525,"port_rcv_data":84154575354792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878076734,"port_rcv_data":3193844083809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627408036017,"port_rcv_data":82214874956673,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394090170,"port_rcv_data":2266575021954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:37.909","gpu_util_pct":99.0,"memory_used_mib":79971.0,"power_w":89.31,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:37.928","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":91.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:37.945","gpu_util_pct":100.0,"memory_used_mib":79917.0,"power_w":90.36,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:37.961","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":92.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:37.977","gpu_util_pct":100.0,"memory_used_mib":79849.0,"power_w":90.11,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:37.991","gpu_util_pct":100.0,"memory_used_mib":79933.0,"power_w":93.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.006","gpu_util_pct":100.0,"memory_used_mib":79917.0,"power_w":91.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:38.020","gpu_util_pct":100.0,"memory_used_mib":79979.0,"power_w":90.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258938.7935376,"monotonic":2345776.242605425,"hcas":{"mlx5_0":{"port_xmit_data":82835871061947,"port_rcv_data":84154853699657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878076734,"port_rcv_data":3193844083809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686371368,"port_rcv_data":82214881281237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394090170,"port_rcv_data":2266575021954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:38.893","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":114.42,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:38.912","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":116.59,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:38.929","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":117.71,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:38.944","gpu_util_pct":14.0,"memory_used_mib":79939.0,"power_w":113.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:38.959","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":111.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:38.973","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":110.13,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:38.987","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":111.68,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:39.001","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":107.25,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258939.7943144,"monotonic":2345777.243382172,"hcas":{"mlx5_0":{"port_xmit_data":82835871061947,"port_rcv_data":84154853699657,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878076734,"port_rcv_data":3193844083809,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686371368,"port_rcv_data":82214881281237,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394090170,"port_rcv_data":2266575021954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:39.893","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":78.5,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:39.911","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:39.928","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.98,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:39.944","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:39.959","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":79.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:39.974","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.95,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:39.988","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:40.002","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":78.07,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258940.7950935,"monotonic":2345778.244161239,"hcas":{"mlx5_0":{"port_xmit_data":82835871071340,"port_rcv_data":84154863272777,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878086143,"port_rcv_data":3193853656929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686380792,"port_rcv_data":82214890854357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394099609,"port_rcv_data":2266584595074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:40.914","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":78.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:40.932","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":81.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:40.949","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.85,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:40.964","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.69,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:40.979","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":79.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:40.994","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:41.008","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.58,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:41.022","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.98,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258941.7958512,"monotonic":2345779.244919035,"hcas":{"mlx5_0":{"port_xmit_data":82835871109098,"port_rcv_data":84154901565257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878123916,"port_rcv_data":3193891949409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686418271,"port_rcv_data":82214929146837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394136902,"port_rcv_data":2266622887554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:41.922","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:41.940","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.6,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:41.957","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.23,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:41.973","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:41.988","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:42.002","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:42.015","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:42.029","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258942.7966006,"monotonic":2345780.245668311,"hcas":{"mlx5_0":{"port_xmit_data":82835871146732,"port_rcv_data":84154939857737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878161488,"port_rcv_data":3193930241889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686455703,"port_rcv_data":82214967439317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394174195,"port_rcv_data":2266661180034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:42.935","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:42.954","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:42.971","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:42.987","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:43.002","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:43.016","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:43.030","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.6,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:43.043","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258943.7973378,"monotonic":2345781.246405535,"hcas":{"mlx5_0":{"port_xmit_data":82835871188768,"port_rcv_data":84154982936777,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878203602,"port_rcv_data":3193973320929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686497810,"port_rcv_data":82215010518357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394216200,"port_rcv_data":2266704259074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:43.939","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:43.957","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:43.975","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:43.991","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.006","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.020","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.034","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:44.048","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258944.798066,"monotonic":2345782.247133534,"hcas":{"mlx5_0":{"port_xmit_data":82835871240228,"port_rcv_data":84155035588937,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878255077,"port_rcv_data":3194025973089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686549623,"port_rcv_data":82215063170517,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394267889,"port_rcv_data":2266756911234,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:44.902","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:44.920","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:44.937","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:44.952","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:44.967","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:44.981","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:44.995","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:45.009","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258945.7988393,"monotonic":2345783.247907057,"hcas":{"mlx5_0":{"port_xmit_data":82835871306661,"port_rcv_data":84155102600777,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878321386,"port_rcv_data":3194092984929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627686616159,"port_rcv_data":82215130182357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394333578,"port_rcv_data":2266823923074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:45.939","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:45.957","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:45.974","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.75,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:45.990","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:46.004","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:46.018","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:46.032","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:46.045","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258946.7995758,"monotonic":2345784.248643606,"hcas":{"mlx5_0":{"port_xmit_data":82835871363066,"port_rcv_data":84155160039497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438878377853,"port_rcv_data":3194150423649,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627687164998,"port_rcv_data":82215214145877,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394877504,"port_rcv_data":2266907886594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:46.944","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:46.962","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:46.979","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:46.995","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.010","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:47.025","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:47.039","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.29,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:47.052","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258947.8003001,"monotonic":2345785.249367878,"hcas":{"mlx5_0":{"port_xmit_data":82835871990041,"port_rcv_data":84155277987165,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438879010594,"port_rcv_data":3194268322019,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627687256650,"port_rcv_data":82215305567701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555394969264,"port_rcv_data":2266999259544,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:47.923","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:47.942","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":79.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:47.960","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.61,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:47.976","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:47.991","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:48.005","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:48.019","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.07,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:48.033","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":78.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258948.8010497,"monotonic":2345786.250117494,"hcas":{"mlx5_0":{"port_xmit_data":82835872240955,"port_rcv_data":84155358671109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438879258578,"port_rcv_data":3194349005963,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627687696540,"port_rcv_data":82215395093245,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555395412579,"port_rcv_data":2267088785088,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:48.934","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:48.953","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:48.970","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.68,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:48.986","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:49.001","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:49.016","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:49.030","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.56,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:49.044","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258949.8017895,"monotonic":2345787.250857129,"hcas":{"mlx5_0":{"port_xmit_data":82835872452204,"port_rcv_data":84155425666683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438879470401,"port_rcv_data":3194416009755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627688243752,"port_rcv_data":82215479947579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555395958613,"port_rcv_data":2267173573720,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:49.947","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":76.96,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:49.966","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.31,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:49.983","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:49.998","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.013","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:50.028","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:50.045","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:50.059","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258950.8025098,"monotonic":2345788.251577669,"hcas":{"mlx5_0":{"port_xmit_data":82835873061478,"port_rcv_data":84155541753212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438880085240,"port_rcv_data":3194532081836,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627688334570,"port_rcv_data":82215569334920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555396049412,"port_rcv_data":2267263018061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:50.928","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":78.17,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:50.946","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:50.964","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:50.980","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":78.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:50.996","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:51.010","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.98,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:51.024","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:51.038","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258951.8031337,"monotonic":2345789.252201152,"hcas":{"mlx5_0":{"port_xmit_data":82835873149177,"port_rcv_data":84155628878146,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438880173047,"port_rcv_data":3194619148520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627688955763,"port_rcv_data":82215686617422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555396664638,"port_rcv_data":2267380232135,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:51.951","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:51.970","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:51.986","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.29,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.002","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.017","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:52.032","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:52.046","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:52.059","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258952.803857,"monotonic":2345790.252924894,"hcas":{"mlx5_0":{"port_xmit_data":82835873527362,"port_rcv_data":84155706466926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438880549217,"port_rcv_data":3194696737378,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627689182001,"port_rcv_data":82215751733134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555396890132,"port_rcv_data":2267445357975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:52.927","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:52.945","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.56,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:52.963","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:52.979","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:52.994","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:53.009","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:53.023","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:53.036","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258953.804604,"monotonic":2345791.253671975,"hcas":{"mlx5_0":{"port_xmit_data":82835873927882,"port_rcv_data":84155805989531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438880954046,"port_rcv_data":3194796135007,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627689618264,"port_rcv_data":82215851254695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555397331107,"port_rcv_data":2267544754984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:53.930","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:53.948","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:53.966","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:53.982","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:53.997","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:54.011","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:54.025","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.8,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:54.039","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258954.8053465,"monotonic":2345792.25441447,"hcas":{"mlx5_0":{"port_xmit_data":82835874310964,"port_rcv_data":84155886504475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438881332060,"port_rcv_data":3194876642549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627689994313,"port_rcv_data":82215931769639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555397707261,"port_rcv_data":2267625262526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:54.927","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:54.945","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:54.962","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.19,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:54.978","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:54.993","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:55.009","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:55.023","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:55.036","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258955.8060932,"monotonic":2345793.255161224,"hcas":{"mlx5_0":{"port_xmit_data":82835874760368,"port_rcv_data":84155996578283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438881780267,"port_rcv_data":3194986684297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627690369304,"port_rcv_data":82216040565539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555398084407,"port_rcv_data":2267734066134,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:55.948","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:55.966","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:55.983","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.7,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:55.998","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.013","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.027","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:56.040","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.05,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:56.053","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258956.8068259,"monotonic":2345794.255893736,"hcas":{"mlx5_0":{"port_xmit_data":82835875123372,"port_rcv_data":84156090557520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438882146423,"port_rcv_data":3195080715672,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627690863243,"port_rcv_data":82216137104236,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555398577338,"port_rcv_data":2267830530121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:56.926","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:56.940","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:56.954","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":76.94,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:56.967","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.98,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:56.981","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":79.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:56.993","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":78.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.006","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.019","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258957.807594,"monotonic":2345795.256661845,"hcas":{"mlx5_0":{"port_xmit_data":82835875572304,"port_rcv_data":84156166515864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438882593759,"port_rcv_data":3195157180867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627691226439,"port_rcv_data":82216214793683,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555398937015,"port_rcv_data":2267908137154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:57.888","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:57.906","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:57.924","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.24,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:57.940","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:57.955","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:57.969","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.48,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:57.983","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":78.9,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:57.997","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.05,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258958.8083847,"monotonic":2345796.257452578,"hcas":{"mlx5_0":{"port_xmit_data":82835876210547,"port_rcv_data":84156315353954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438883232018,"port_rcv_data":3195306815336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627691510755,"port_rcv_data":82216351833794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555399222435,"port_rcv_data":2268045243449,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:58.935","gpu_util_pct":0.0,"memory_used_mib":79973.0,"power_w":77.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:58.954","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":80.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:58.972","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":77.26,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:58.988","gpu_util_pct":0.0,"memory_used_mib":79939.0,"power_w":77.97,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.003","gpu_util_pct":0.0,"memory_used_mib":79851.0,"power_w":78.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:35:59.018","gpu_util_pct":0.0,"memory_used_mib":79935.0,"power_w":77.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:35:59.032","gpu_util_pct":0.0,"memory_used_mib":79919.0,"power_w":79.04,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:35:59.045","gpu_util_pct":0.0,"memory_used_mib":79981.0,"power_w":77.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258959.8091285,"monotonic":2345797.258196236,"hcas":{"mlx5_0":{"port_xmit_data":82835879885024,"port_rcv_data":84156516843312,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438883453606,"port_rcv_data":3195360064473,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82627839664614,"port_rcv_data":82216418787110,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555399783938,"port_rcv_data":2268108683150,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:35:59.925","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":96.42,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:35:59.945","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":99.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:35:59.962","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":96.92,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:35:59.979","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":99.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:35:59.994","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":99.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.009","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":98.5,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.024","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":99.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:00.038","gpu_util_pct":100.0,"memory_used_mib":79981.0,"power_w":97.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258960.809885,"monotonic":2345798.258952558,"hcas":{"mlx5_0":{"port_xmit_data":82835887326093,"port_rcv_data":84156910317348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438883866511,"port_rcv_data":3195446331453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82628147302913,"port_rcv_data":82216512071276,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555400220279,"port_rcv_data":2268194948830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:00.894","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":121.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:00.913","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":120.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:00.931","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.16,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:00.947","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":120.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:00.962","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":124.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:00.977","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":123.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:00.992","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":122.14,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:01.006","gpu_util_pct":99.0,"memory_used_mib":79981.0,"power_w":119.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258961.8106654,"monotonic":2345799.259733109,"hcas":{"mlx5_0":{"port_xmit_data":82835894797759,"port_rcv_data":84157311404916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438884292063,"port_rcv_data":3195538943035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82628455936148,"port_rcv_data":82216619319352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555400747341,"port_rcv_data":2268295163872,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:01.913","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":122.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:01.932","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":124.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:01.950","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":121.43,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:01.966","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":122.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:01.982","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":120.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:01.997","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":123.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:02.011","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.42,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:02.025","gpu_util_pct":100.0,"memory_used_mib":79981.0,"power_w":120.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258962.8109593,"monotonic":2345800.26002738,"hcas":{"mlx5_0":{"port_xmit_data":82835902353747,"port_rcv_data":84157722569961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438884872042,"port_rcv_data":3195642713739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82628763419591,"port_rcv_data":82216713713364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555401067493,"port_rcv_data":2268382487044,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:02.913","gpu_util_pct":99.0,"memory_used_mib":79973.0,"power_w":120.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:02.932","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":121.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:02.949","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":120.91,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:02.966","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":120.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:02.981","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":122.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:02.996","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":122.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:03.010","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":120.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:03.024","gpu_util_pct":100.0,"memory_used_mib":79981.0,"power_w":122.2,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258963.8117313,"monotonic":2345801.260799298,"hcas":{"mlx5_0":{"port_xmit_data":82835909961720,"port_rcv_data":84158102728503,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438885459027,"port_rcv_data":3195714159833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82629072164180,"port_rcv_data":82216783393479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555401472989,"port_rcv_data":2268445090366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:03.904","gpu_util_pct":99.0,"memory_used_mib":79973.0,"power_w":124.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:03.922","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":126.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:03.940","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":119.5,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:03.957","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":121.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:03.972","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":121.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:03.987","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":125.33,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:04.001","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.13,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:04.015","gpu_util_pct":99.0,"memory_used_mib":79981.0,"power_w":120.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258964.8125067,"monotonic":2345802.261574048,"hcas":{"mlx5_0":{"port_xmit_data":82835918263782,"port_rcv_data":84158559911170,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438885734586,"port_rcv_data":3195829217447,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82629415023760,"port_rcv_data":82216933016287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555402237139,"port_rcv_data":2268586671116,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:04.912","gpu_util_pct":90.0,"memory_used_mib":79973.0,"power_w":122.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:04.931","gpu_util_pct":90.0,"memory_used_mib":79935.0,"power_w":125.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:04.949","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":122.24,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:04.965","gpu_util_pct":91.0,"memory_used_mib":79939.0,"power_w":122.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:04.981","gpu_util_pct":91.0,"memory_used_mib":79851.0,"power_w":120.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:04.996","gpu_util_pct":91.0,"memory_used_mib":79935.0,"power_w":122.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:05.010","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":123.69,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:05.024","gpu_util_pct":100.0,"memory_used_mib":79981.0,"power_w":117.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258965.813272,"monotonic":2345803.262339757,"hcas":{"mlx5_0":{"port_xmit_data":82835930356351,"port_rcv_data":84159070380423,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438886200160,"port_rcv_data":3195855742247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82629899260118,"port_rcv_data":82216962299646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555402546627,"port_rcv_data":2268604354316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:05.933","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":125.07,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:05.951","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":128.83,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:05.968","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.1,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:05.984","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":126.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:05.999","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":123.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:06.014","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:06.028","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.11,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:06.042","gpu_util_pct":100.0,"memory_used_mib":79981.0,"power_w":120.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258966.814019,"monotonic":2345804.263086873,"hcas":{"mlx5_0":{"port_xmit_data":82835942281580,"port_rcv_data":84159571988047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438886519522,"port_rcv_data":3195874409488,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82630383478694,"port_rcv_data":82216991566788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555402855806,"port_rcv_data":2268622037516,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:06.916","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":126.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:06.935","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:06.953","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":125.46,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:06.969","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":126.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:06.984","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":124.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:07.000","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.48,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:07.014","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":124.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:07.028","gpu_util_pct":100.0,"memory_used_mib":79981.0,"power_w":119.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258967.8147845,"monotonic":2345805.263852256,"hcas":{"mlx5_0":{"port_xmit_data":82835954402035,"port_rcv_data":84160086468926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438886891010,"port_rcv_data":3195897786403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82630875764624,"port_rcv_data":82217025100587,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555403225419,"port_rcv_data":2268643931420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:07.917","gpu_util_pct":100.0,"memory_used_mib":79973.0,"power_w":126.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:07.936","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":128.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:07.953","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":122.63,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:07.969","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":124.04,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:07.985","gpu_util_pct":100.0,"memory_used_mib":79851.0,"power_w":125.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:08.000","gpu_util_pct":100.0,"memory_used_mib":79935.0,"power_w":127.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:08.014","gpu_util_pct":100.0,"memory_used_mib":79919.0,"power_w":127.66,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:08.029","gpu_util_pct":100.0,"memory_used_mib":79981.0,"power_w":123.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258968.8149564,"monotonic":2345806.264024438,"hcas":{"mlx5_0":{"port_xmit_data":82835966495505,"port_rcv_data":84160596985978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438887347454,"port_rcv_data":3195921845323,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82631361421992,"port_rcv_data":82217054355997,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555403531513,"port_rcv_data":2268661614620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:08.937","gpu_util_pct":92.0,"memory_used_mib":79975.0,"power_w":124.43,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:08.955","gpu_util_pct":95.0,"memory_used_mib":79937.0,"power_w":126.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:08.973","gpu_util_pct":93.0,"memory_used_mib":79921.0,"power_w":124.03,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:08.989","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":125.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.005","gpu_util_pct":93.0,"memory_used_mib":79853.0,"power_w":121.97,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.019","gpu_util_pct":94.0,"memory_used_mib":79937.0,"power_w":127.58,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:09.034","gpu_util_pct":93.0,"memory_used_mib":79921.0,"power_w":125.38,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:09.048","gpu_util_pct":100.0,"memory_used_mib":79983.0,"power_w":119.33,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258969.8156981,"monotonic":2345807.264766061,"hcas":{"mlx5_0":{"port_xmit_data":82835978924342,"port_rcv_data":84161246710710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438887507352,"port_rcv_data":3195930687785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82632002305313,"port_rcv_data":82217101418642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555404123535,"port_rcv_data":2268696427888,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:09.916","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":127.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:09.934","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":129.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:09.952","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":127.71,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:09.968","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":129.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:09.983","gpu_util_pct":100.0,"memory_used_mib":79853.0,"power_w":124.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:09.998","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":130.21,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.013","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":129.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:10.027","gpu_util_pct":100.0,"memory_used_mib":79983.0,"power_w":123.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258970.8164628,"monotonic":2345808.265530675,"hcas":{"mlx5_0":{"port_xmit_data":82835991699878,"port_rcv_data":84161922237856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438888102583,"port_rcv_data":3195965501639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82632643178882,"port_rcv_data":82217122441779,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555404276489,"port_rcv_data":2268705269764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:10.898","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":126.57,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:10.918","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":128.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:10.935","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":129.33,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:10.952","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":130.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:10.967","gpu_util_pct":100.0,"memory_used_mib":79853.0,"power_w":126.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:10.982","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":128.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:10.996","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":129.78,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:11.010","gpu_util_pct":100.0,"memory_used_mib":79983.0,"power_w":124.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258971.817245,"monotonic":2345809.266312826,"hcas":{"mlx5_0":{"port_xmit_data":82836004478725,"port_rcv_data":84162600192698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438888462106,"port_rcv_data":3195989130359,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82633297858036,"port_rcv_data":82217152537174,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555404589186,"port_rcv_data":2268722952964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:11.916","gpu_util_pct":100.0,"memory_used_mib":79975.0,"power_w":130.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:11.935","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":129.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:11.952","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":126.98,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:11.969","gpu_util_pct":100.0,"memory_used_mib":79941.0,"power_w":130.81,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:11.984","gpu_util_pct":100.0,"memory_used_mib":79853.0,"power_w":124.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:11.999","gpu_util_pct":100.0,"memory_used_mib":79937.0,"power_w":130.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:12.013","gpu_util_pct":100.0,"memory_used_mib":79921.0,"power_w":131.56,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:12.026","gpu_util_pct":100.0,"memory_used_mib":79983.0,"power_w":127.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258972.818012,"monotonic":2345810.267079901,"hcas":{"mlx5_0":{"port_xmit_data":82836016892423,"port_rcv_data":84163251493256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438888725885,"port_rcv_data":3196000868039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82633937848981,"port_rcv_data":82217191216100,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555405053861,"port_rcv_data":2268749477764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:12.943","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":130.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:12.962","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":129.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:12.980","gpu_util_pct":98.0,"memory_used_mib":79923.0,"power_w":127.22,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:12.997","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.85,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.012","gpu_util_pct":98.0,"memory_used_mib":79855.0,"power_w":127.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:13.028","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.48,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:13.042","gpu_util_pct":97.0,"memory_used_mib":79923.0,"power_w":129.43,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:13.056","gpu_util_pct":100.0,"memory_used_mib":79985.0,"power_w":126.5,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258973.8187559,"monotonic":2345811.267823614,"hcas":{"mlx5_0":{"port_xmit_data":82836028855483,"port_rcv_data":84164046662492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438889192900,"port_rcv_data":3196027392839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82634707213854,"port_rcv_data":82217220444253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555405366666,"port_rcv_data":2268767160964,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:13.921","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":128.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:13.940","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":135.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:13.958","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":128.35,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:13.974","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:13.990","gpu_util_pct":100.0,"memory_used_mib":79855.0,"power_w":128.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.005","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":133.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.020","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:14.034","gpu_util_pct":100.0,"memory_used_mib":79985.0,"power_w":130.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258974.8195088,"monotonic":2345812.268576582,"hcas":{"mlx5_0":{"port_xmit_data":82836040774148,"port_rcv_data":84164844487935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438889664797,"port_rcv_data":3196055023731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82635477417238,"port_rcv_data":82217250735755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555405679084,"port_rcv_data":2268785949084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:14.891","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":129.84,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:14.911","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":134.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:14.929","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.25,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:14.945","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:14.961","gpu_util_pct":100.0,"memory_used_mib":79855.0,"power_w":127.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:14.976","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":129.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:14.990","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:15.005","gpu_util_pct":100.0,"memory_used_mib":79985.0,"power_w":126.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258975.820294,"monotonic":2345813.269361656,"hcas":{"mlx5_0":{"port_xmit_data":82836052641259,"port_rcv_data":84165642851959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438889815829,"port_rcv_data":3196063797119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82636267689223,"port_rcv_data":82217298427685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406284964,"port_rcv_data":2268821867536,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:15.920","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":128.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:15.939","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":131.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:15.956","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":128.12,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:15.972","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:15.987","gpu_util_pct":100.0,"memory_used_mib":79855.0,"power_w":130.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:16.004","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":132.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:16.018","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":132.01,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:16.032","gpu_util_pct":100.0,"memory_used_mib":79985.0,"power_w":128.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258976.8210523,"monotonic":2345814.270120152,"hcas":{"mlx5_0":{"port_xmit_data":82836064530085,"port_rcv_data":84166439822183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890281124,"port_rcv_data":3196090943846,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82637037836467,"port_rcv_data":82217326057035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406534777,"port_rcv_data":2268838027862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:16.920","gpu_util_pct":100.0,"memory_used_mib":79977.0,"power_w":131.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:16.939","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":133.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:16.957","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":130.82,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:16.974","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:16.990","gpu_util_pct":100.0,"memory_used_mib":79855.0,"power_w":128.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:17.005","gpu_util_pct":100.0,"memory_used_mib":79939.0,"power_w":133.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:17.019","gpu_util_pct":100.0,"memory_used_mib":79923.0,"power_w":131.84,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:17.034","gpu_util_pct":100.0,"memory_used_mib":79985.0,"power_w":128.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258977.821812,"monotonic":2345815.270879685,"hcas":{"mlx5_0":{"port_xmit_data":82836076477200,"port_rcv_data":84167244441901,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890600486,"port_rcv_data":3196111375672,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82637824376057,"port_rcv_data":82217357043921,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406884426,"port_rcv_data":2268857233810,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:17.926","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:17.945","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:17.962","gpu_util_pct":91.0,"memory_used_mib":79943.0,"power_w":126.08,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:17.978","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:17.994","gpu_util_pct":96.0,"memory_used_mib":79875.0,"power_w":125.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:18.008","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.35,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:18.023","gpu_util_pct":91.0,"memory_used_mib":79943.0,"power_w":131.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:18.037","gpu_util_pct":91.0,"memory_used_mib":80005.0,"power_w":133.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258978.8225698,"monotonic":2345816.271637544,"hcas":{"mlx5_0":{"port_xmit_data":82836087060661,"port_rcv_data":84168063938279,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82638627408550,"port_rcv_data":82217368821985,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:18.924","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:18.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:18.961","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.62,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:18.978","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:18.994","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.009","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:19.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.67,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:19.038","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.35,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258979.823326,"monotonic":2345817.272394129,"hcas":{"mlx5_0":{"port_xmit_data":82836097461303,"port_rcv_data":84168886090049,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82639447920045,"port_rcv_data":82217379297619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:19.905","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:19.924","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:19.942","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.13,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:19.958","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:19.974","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:19.989","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:20.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.23,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:20.017","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258980.8240893,"monotonic":2345818.273157246,"hcas":{"mlx5_0":{"port_xmit_data":82836107376907,"port_rcv_data":84169771065853,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82640335520629,"port_rcv_data":82217389355033,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:20.924","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.25,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:20.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:20.961","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.4,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:20.978","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:20.993","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.008","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.45,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:21.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:21.037","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258981.8248456,"monotonic":2345819.273913648,"hcas":{"mlx5_0":{"port_xmit_data":82836117339111,"port_rcv_data":84170679228183,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82641243700334,"port_rcv_data":82217399441780,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:21.906","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:21.925","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:21.943","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.34,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:21.960","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:21.975","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:21.990","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:22.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.11,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:22.019","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.73,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258982.8256168,"monotonic":2345820.274684797,"hcas":{"mlx5_0":{"port_xmit_data":82836126171160,"port_rcv_data":84171573187904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82642138072595,"port_rcv_data":82217408404679,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:22.929","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:22.948","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:22.965","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.93,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:22.981","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:22.997","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:23.012","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.18,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:23.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.32,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:23.044","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258983.8263726,"monotonic":2345821.275440544,"hcas":{"mlx5_0":{"port_xmit_data":82836134452177,"port_rcv_data":84172497265193,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82643058835039,"port_rcv_data":82217416870862,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:23.929","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":128.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:23.948","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:23.966","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":123.3,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:23.983","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":125.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:23.999","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":126.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:24.016","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.98,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:24.031","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.55,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:24.047","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":125.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258984.8271358,"monotonic":2345822.276203265,"hcas":{"mlx5_0":{"port_xmit_data":82836142076549,"port_rcv_data":84173350002789,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82643914121362,"port_rcv_data":82217424775463,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:24.946","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":129.21,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:24.965","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:24.982","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.35,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:24.999","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":123.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.015","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":126.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:25.030","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.45,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:25.045","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.43,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:25.058","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":126.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258985.8278868,"monotonic":2345823.27695443,"hcas":{"mlx5_0":{"port_xmit_data":82836150210649,"port_rcv_data":84174256337994,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82644820434612,"port_rcv_data":82217433096505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:25.916","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":129.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:25.935","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":127.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:25.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.74,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:25.969","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:25.984","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":122.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:26.003","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:26.018","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.57,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:26.031","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":123.97,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258986.8286562,"monotonic":2345824.277723806,"hcas":{"mlx5_0":{"port_xmit_data":82836157991990,"port_rcv_data":84175112233858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82645675126028,"port_rcv_data":82217441096603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:26.932","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:26.951","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:26.968","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":123.72,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:26.985","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.000","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.016","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:27.031","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.31,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:27.045","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258987.8294203,"monotonic":2345825.278488085,"hcas":{"mlx5_0":{"port_xmit_data":82836166556506,"port_rcv_data":84176070429971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890894490,"port_rcv_data":3196127660095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82646638775300,"port_rcv_data":82217449855521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406895292,"port_rcv_data":2268858581932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:27.914","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:27.933","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:27.950","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.53,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:27.966","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":125.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:27.982","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:27.997","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:28.013","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:28.027","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.71,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258988.8301995,"monotonic":2345826.279267353,"hcas":{"mlx5_0":{"port_xmit_data":82836175579045,"port_rcv_data":84176999863034,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890903898,"port_rcv_data":3196137233215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82647555278768,"port_rcv_data":82217468625383,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406904778,"port_rcv_data":2268868155052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:28.969","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:28.988","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.77,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.022","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.038","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:29.068","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:29.082","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258989.8309193,"monotonic":2345827.27998727,"hcas":{"mlx5_0":{"port_xmit_data":82836184295434,"port_rcv_data":84177897981170,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890917817,"port_rcv_data":3196151592895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82648437635939,"port_rcv_data":82217491847543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406918790,"port_rcv_data":2268882514732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:29.914","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:29.933","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:29.951","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.33,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:29.968","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:29.983","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:29.999","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.47,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:30.014","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.44,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:30.028","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258990.8316936,"monotonic":2345828.280761521,"hcas":{"mlx5_0":{"port_xmit_data":82836193770156,"port_rcv_data":84178892626297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890945826,"port_rcv_data":3196180312255,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82649403070718,"port_rcv_data":82217530162028,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406946783,"port_rcv_data":2268911234092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:30.936","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.07,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:30.955","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:30.973","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.71,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:30.990","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.005","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":125.38,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:31.020","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:31.037","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.81,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:31.052","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258991.832445,"monotonic":2345829.281512889,"hcas":{"mlx5_0":{"port_xmit_data":82836202972152,"port_rcv_data":84179845647935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890959946,"port_rcv_data":3196194671935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82650343248385,"port_rcv_data":82217553884299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406961105,"port_rcv_data":2268925593772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:31.917","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.07,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:31.936","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.61,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:31.953","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.59,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:31.969","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:31.985","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":126.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:32.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:32.014","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.11,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:32.030","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258992.8332207,"monotonic":2345830.282288607,"hcas":{"mlx5_0":{"port_xmit_data":82836211830284,"port_rcv_data":84180760504011,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890973927,"port_rcv_data":3196209031615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82651242779552,"port_rcv_data":82217577263647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406975117,"port_rcv_data":2268939953452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:32.952","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":130.06,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:32.971","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:32.989","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.04,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.005","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.021","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:33.038","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:33.053","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.4,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:33.067","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":125.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258993.8339622,"monotonic":2345831.28302987,"hcas":{"mlx5_0":{"port_xmit_data":82836221346796,"port_rcv_data":84181745818556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438890987955,"port_rcv_data":3196223391295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82652214355630,"port_rcv_data":82217601266797,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406989129,"port_rcv_data":2268954313132,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:33.937","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:33.956","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.07,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:33.969","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.69,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:33.983","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:33.997","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.22,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:34.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:34.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.15,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:34.036","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258994.8347328,"monotonic":2345832.283800873,"hcas":{"mlx5_0":{"port_xmit_data":82836230840190,"port_rcv_data":84182686623395,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438891513761,"port_rcv_data":3196249916095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82653129659765,"port_rcv_data":82217610418953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555406989129,"port_rcv_data":2268954313132,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:34.937","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:34.955","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:34.973","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":123.04,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:34.990","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":123.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:35.005","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:35.021","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.86,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:35.035","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.1,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:35.050","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258995.8354897,"monotonic":2345833.284557786,"hcas":{"mlx5_0":{"port_xmit_data":82836240093692,"port_rcv_data":84183711970404,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438891786143,"port_rcv_data":3196360030467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82654046238424,"port_rcv_data":82217747443090,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555407562458,"port_rcv_data":2269082110576,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:35.936","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.25,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:35.955","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:35.973","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.66,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:35.989","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:36.004","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:36.019","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:36.034","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:36.048","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258996.83625,"monotonic":2345834.285317958,"hcas":{"mlx5_0":{"port_xmit_data":82836250180471,"port_rcv_data":84184715889456,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396238,"port_rcv_data":3196395949505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82655014332815,"port_rcv_data":82217765899379,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555407718202,"port_rcv_data":2269090952452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:36.942","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:36.961","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.26,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:36.978","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.81,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:36.995","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.010","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:37.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:37.040","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:37.054","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258997.8370016,"monotonic":2345835.286069421,"hcas":{"mlx5_0":{"port_xmit_data":82836259160260,"port_rcv_data":84185631949008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82655931583384,"port_rcv_data":82217810968009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:37.920","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.4,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:37.939","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:37.957","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.55,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:37.973","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:37.989","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:38.005","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.32,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:38.019","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.69,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:38.034","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258998.8377743,"monotonic":2345836.286842352,"hcas":{"mlx5_0":{"port_xmit_data":82836268450776,"port_rcv_data":84186582260697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82656880853247,"port_rcv_data":82217820413108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:38.957","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:38.976","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.47,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:38.994","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.16,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.011","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:39.027","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:39.042","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:39.057","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.32,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:39.071","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.75,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788258999.838511,"monotonic":2345837.287578956,"hcas":{"mlx5_0":{"port_xmit_data":82836276361043,"port_rcv_data":84187443256243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82657743512306,"port_rcv_data":82217828559571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:39.941","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":127.83,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:39.960","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:39.977","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":122.96,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:39.994","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":123.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:40.009","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":124.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:40.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:40.040","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:40.054","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":126.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259000.8392673,"monotonic":2345838.288335115,"hcas":{"mlx5_0":{"port_xmit_data":82836284097500,"port_rcv_data":84188301475531,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82658601802387,"port_rcv_data":82217836507804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:40.940","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":127.54,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:40.959","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":125.21,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:40.977","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.14,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:40.993","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":124.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:41.009","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":125.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:41.024","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:41.038","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:41.052","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259001.8400252,"monotonic":2345839.289092904,"hcas":{"mlx5_0":{"port_xmit_data":82836292499928,"port_rcv_data":84189239133416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82659538284451,"port_rcv_data":82217845087967,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:41.964","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:41.983","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.63,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.001","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.28,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.017","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":123.19,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.033","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:42.048","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:42.063","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.2,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:42.077","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259002.840756,"monotonic":2345840.289823795,"hcas":{"mlx5_0":{"port_xmit_data":82836300576765,"port_rcv_data":84190138719599,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82660439747212,"port_rcv_data":82217853379698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:42.923","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.99,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:42.943","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:42.960","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.02,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:42.976","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:42.991","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:43.009","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.12,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:43.026","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.34,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:43.040","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259003.8415132,"monotonic":2345841.29058107,"hcas":{"mlx5_0":{"port_xmit_data":82836308825527,"port_rcv_data":84191058771881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82661357063335,"port_rcv_data":82217861807568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:43.963","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.17,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:43.981","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:43.998","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":121.96,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.014","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":125.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.030","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:44.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:44.062","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:44.076","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":126.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259004.8422441,"monotonic":2345842.29131191,"hcas":{"mlx5_0":{"port_xmit_data":82836316536650,"port_rcv_data":84191918244393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82662217006665,"port_rcv_data":82217869756481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:44.927","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:44.946","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:44.963","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":120.49,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:44.979","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":123.22,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:44.995","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":125.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:45.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.38,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:45.025","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.25,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:45.039","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":126.01,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259005.8430338,"monotonic":2345843.292101432,"hcas":{"mlx5_0":{"port_xmit_data":82836324838343,"port_rcv_data":84192845262252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82663146053224,"port_rcv_data":82217878302578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:45.926","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":129.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:45.944","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:45.962","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":122.74,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:45.979","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:45.994","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:46.009","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:46.024","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.56,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:46.039","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259006.8438213,"monotonic":2345844.292888841,"hcas":{"mlx5_0":{"port_xmit_data":82836332706529,"port_rcv_data":84193722673518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82664023723871,"port_rcv_data":82217886405679,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:46.929","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":129.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:46.949","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:46.966","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.17,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:46.983","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:46.999","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:47.014","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.77,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:47.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.55,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:47.043","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259007.844595,"monotonic":2345845.293662676,"hcas":{"mlx5_0":{"port_xmit_data":82836340800779,"port_rcv_data":84194625706915,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82664925674054,"port_rcv_data":82217894686891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:47.929","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.39,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:47.947","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:47.965","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.17,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:47.981","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:47.996","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":125.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:48.012","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:48.026","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:48.042","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":122.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259008.8453658,"monotonic":2345846.294433389,"hcas":{"mlx5_0":{"port_xmit_data":82836348683130,"port_rcv_data":84195494589210,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892396424,"port_rcv_data":3196395950367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82665796461744,"port_rcv_data":82217902820638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408328096,"port_rcv_data":2269126870904,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:48.948","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.45,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:48.967","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.38,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:48.984","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.2,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:49.000","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":122.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:49.016","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:49.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:49.046","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.17,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:49.060","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":125.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259009.8461084,"monotonic":2345847.295176274,"hcas":{"mlx5_0":{"port_xmit_data":82836357919484,"port_rcv_data":84196446364075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892405848,"port_rcv_data":3196405523487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82666739206418,"port_rcv_data":82217921782633,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408337598,"port_rcv_data":2269136444024,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:49.971","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:49.996","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:50.014","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.31,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:50.030","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:50.046","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:50.062","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:50.076","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.06,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:50.090","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259010.8468473,"monotonic":2345848.295913633,"hcas":{"mlx5_0":{"port_xmit_data":82836367178423,"port_rcv_data":84197416881023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892432756,"port_rcv_data":3196433279929,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82667680656600,"port_rcv_data":82217959882108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408365622,"port_rcv_data":2269165163384,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:50.968","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.69,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:50.989","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.76,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:51.022","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:51.038","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:51.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.63,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:51.067","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:51.081","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259011.8469672,"monotonic":2345849.296034985,"hcas":{"mlx5_0":{"port_xmit_data":82836376579317,"port_rcv_data":84198394602656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892447869,"port_rcv_data":3196448602527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82668642756721,"port_rcv_data":82217983793795,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408379572,"port_rcv_data":2269179523064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:51.949","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:51.968","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:51.986","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.52,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:52.002","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:52.018","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:52.033","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:52.049","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.06,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:52.063","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259012.8477294,"monotonic":2345850.296796942,"hcas":{"mlx5_0":{"port_xmit_data":82836385992299,"port_rcv_data":84199368267653,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892461865,"port_rcv_data":3196462962207,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82669602460780,"port_rcv_data":82218007698057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408393661,"port_rcv_data":2269193882744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:52.950","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:52.969","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.15,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:52.986","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.15,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:53.002","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:53.018","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:53.032","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.26,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:53.047","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:53.061","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259013.8484986,"monotonic":2345851.297566301,"hcas":{"mlx5_0":{"port_xmit_data":82836395403475,"port_rcv_data":84200345056256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892475862,"port_rcv_data":3196477321887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82670566431126,"port_rcv_data":82218031627792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408407627,"port_rcv_data":2269208242424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:53.948","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:53.966","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:53.985","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.59,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:54.001","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:54.017","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:54.032","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:54.047","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.82,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:54.062","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.03,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259014.8492637,"monotonic":2345852.298331353,"hcas":{"mlx5_0":{"port_xmit_data":82836404683467,"port_rcv_data":84201305847474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892489998,"port_rcv_data":3196491681567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82671510671751,"port_rcv_data":82218055390124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408421825,"port_rcv_data":2269222602104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:54.969","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.43,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:54.988","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.38,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:55.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.49,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:55.023","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.09,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:55.038","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:55.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.55,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:55.068","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:55.083","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259015.8500063,"monotonic":2345853.299073977,"hcas":{"mlx5_0":{"port_xmit_data":82836414074192,"port_rcv_data":84202256170391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892660653,"port_rcv_data":3196500523167,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82672455360572,"port_rcv_data":82218073642040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555408603609,"port_rcv_data":2269231443704,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:55.969","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.98,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:55.989","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:56.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.91,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:56.023","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:56.039","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.43,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:56.055","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.77,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:56.070","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.57,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:56.084","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259016.8507514,"monotonic":2345854.299819299,"hcas":{"mlx5_0":{"port_xmit_data":82836423990190,"port_rcv_data":84203354360142,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438892943838,"port_rcv_data":3196610637539,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82673443447480,"port_rcv_data":82218211282878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409188765,"port_rcv_data":2269359241148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:56.969","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:56.987","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.02,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:57.022","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:57.037","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:57.052","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.84,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:57.067","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.3,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:57.081","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259017.8514962,"monotonic":2345855.30056382,"hcas":{"mlx5_0":{"port_xmit_data":82836433438161,"port_rcv_data":84204303400172,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893256922,"port_rcv_data":3196628320954,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82674374490037,"port_rcv_data":82218229545020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409346431,"port_rcv_data":2269368220761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:57.952","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.67,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:57.971","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.38,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:57.989","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.81,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:58.004","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:58.020","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:58.036","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.27,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:58.050","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.88,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:58.064","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259018.85226,"monotonic":2345856.301327733,"hcas":{"mlx5_0":{"port_xmit_data":82836442844818,"port_rcv_data":84205248800152,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893565791,"port_rcv_data":3196646280689,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82675301347078,"port_rcv_data":82218247779906,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409501694,"port_rcv_data":2269377200512,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:58.952","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:58.971","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:58.988","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.78,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.004","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:36:59.020","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.53,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:36:59.035","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.9,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:36:59.050","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:36:59.064","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259019.8530233,"monotonic":2345857.302090971,"hcas":{"mlx5_0":{"port_xmit_data":82836452255178,"port_rcv_data":84206204784878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893776560,"port_rcv_data":3196661380370,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82676242071709,"port_rcv_data":82218275116561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409813523,"port_rcv_data":2269395159807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:36:59.939","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.48,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:36:59.958","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:36:59.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.46,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:36:59.991","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:00.007","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:00.022","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:00.037","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.39,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:00.051","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.61,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259020.8538105,"monotonic":2345858.302878224,"hcas":{"mlx5_0":{"port_xmit_data":82836460842747,"port_rcv_data":84207083461254,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893872753,"port_rcv_data":3196664240639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82677118749792,"port_rcv_data":82218283819988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409813554,"port_rcv_data":2269395159876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:00.938","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":129.36,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:00.957","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:00.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":123.53,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:00.991","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:01.007","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":124.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:01.022","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.11,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:01.036","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.14,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:01.050","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259021.8545954,"monotonic":2345859.30366311,"hcas":{"mlx5_0":{"port_xmit_data":82836469277131,"port_rcv_data":84208006762297,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893872753,"port_rcv_data":3196664240639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82678044152088,"port_rcv_data":82218292484113,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409813554,"port_rcv_data":2269395159876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:01.957","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.26,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:01.976","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:01.993","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":122.45,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.009","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:02.024","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":123.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:02.039","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.77,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:02.054","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.91,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:02.067","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.91,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259022.8549576,"monotonic":2345860.304025289,"hcas":{"mlx5_0":{"port_xmit_data":82836478418835,"port_rcv_data":84208948485819,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893882363,"port_rcv_data":3196673813759,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82678976834403,"port_rcv_data":82218311364299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409822963,"port_rcv_data":2269404732996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:02.939","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:02.957","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:02.975","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.74,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:02.991","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:03.006","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:03.021","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:03.037","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.64,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:03.051","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259023.8557415,"monotonic":2345861.304809235,"hcas":{"mlx5_0":{"port_xmit_data":82836487725536,"port_rcv_data":84209926110815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893910464,"port_rcv_data":3196702533119,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82679924969647,"port_rcv_data":82218349516918,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409851095,"port_rcv_data":2269433452356,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:03.961","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.54,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:03.980","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.87,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:03.997","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.02,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:04.013","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:04.028","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:04.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:04.058","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:04.072","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259024.856502,"monotonic":2345862.305569765,"hcas":{"mlx5_0":{"port_xmit_data":82836496835835,"port_rcv_data":84210868892841,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893924476,"port_rcv_data":3196716892799,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82680854471964,"port_rcv_data":82218373150097,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409865045,"port_rcv_data":2269447812036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:04.960","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:04.979","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.94,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:04.996","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.51,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.012","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:05.028","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:05.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.95,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:05.058","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.03,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:05.072","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259025.8572664,"monotonic":2345863.306334244,"hcas":{"mlx5_0":{"port_xmit_data":82836505984562,"port_rcv_data":84211817028615,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893938566,"port_rcv_data":3196731252479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82681786483830,"port_rcv_data":82218396808656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409879057,"port_rcv_data":2269462171716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:05.939","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:05.958","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.11,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:05.976","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.23,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:05.992","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:06.008","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":124.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:06.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:06.037","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.21,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:06.051","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259026.8580601,"monotonic":2345864.307127845,"hcas":{"mlx5_0":{"port_xmit_data":82836514844116,"port_rcv_data":84212733750584,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893952562,"port_rcv_data":3196745612159,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82682690382904,"port_rcv_data":82218420216703,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409893069,"port_rcv_data":2269476531396,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:06.941","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:06.959","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.28,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:06.977","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.03,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:06.994","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:07.010","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:07.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:07.039","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:07.054","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259027.8588393,"monotonic":2345865.307907014,"hcas":{"mlx5_0":{"port_xmit_data":82836524351873,"port_rcv_data":84213719230476,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893967427,"port_rcv_data":3196759971839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82683660072250,"port_rcv_data":82218444198909,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555409907221,"port_rcv_data":2269490891076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:07.941","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.84,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:07.960","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:07.977","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.04,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:07.994","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:08.009","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:08.024","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:08.039","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.55,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:08.054","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259028.8596203,"monotonic":2345866.308688142,"hcas":{"mlx5_0":{"port_xmit_data":82836533578916,"port_rcv_data":84214662542281,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438893967427,"port_rcv_data":3196759971839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82684605556437,"port_rcv_data":82218480141864,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555410421805,"port_rcv_data":2269517415876,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:08.942","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.6,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:08.961","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.58,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:08.979","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.72,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:08.995","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:09.010","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:09.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:09.040","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.34,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:09.054","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259029.8604026,"monotonic":2345867.309470605,"hcas":{"mlx5_0":{"port_xmit_data":82836543680310,"port_rcv_data":84215771393314,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438894397350,"port_rcv_data":3196877433159,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82685598077799,"port_rcv_data":82218598638912,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555410686018,"port_rcv_data":2269626033252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:09.976","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:09.995","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:10.012","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.25,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:10.028","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:10.043","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:10.062","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.99,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:10.076","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.31,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:10.090","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.17,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259030.8611474,"monotonic":2345868.310215215,"hcas":{"mlx5_0":{"port_xmit_data":82836553505949,"port_rcv_data":84216761831624,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438894721114,"port_rcv_data":3196895116574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82686568362108,"port_rcv_data":82218625524417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411004636,"port_rcv_data":2269643509095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:10.964","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:10.983","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:11.001","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.15,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:11.017","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:11.033","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:11.048","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.92,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:11.063","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.03,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:11.077","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259031.861909,"monotonic":2345869.310976896,"hcas":{"mlx5_0":{"port_xmit_data":82836562535483,"port_rcv_data":84217676642603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438894877726,"port_rcv_data":3196903958832,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82687477049083,"port_rcv_data":82218652716691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411314156,"port_rcv_data":2269661399181,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:11.967","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.72,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:11.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.18,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.34,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:12.020","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":124.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:12.035","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:12.050","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:12.064","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.96,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:12.078","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.19,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259032.8626642,"monotonic":2345870.311731975,"hcas":{"mlx5_0":{"port_xmit_data":82836572117480,"port_rcv_data":84218631711618,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895347190,"port_rcv_data":3196930483667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82688405780159,"port_rcv_data":82218661991590,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411314249,"port_rcv_data":2269661399388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:12.947","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.89,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:12.966","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.12,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:12.983","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.35,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:13.000","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:13.016","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:13.030","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.02,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:13.045","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.76,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:13.059","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259033.8634439,"monotonic":2345871.312511738,"hcas":{"mlx5_0":{"port_xmit_data":82836581152327,"port_rcv_data":84219575401042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895347190,"port_rcv_data":3196930483667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82689348029560,"port_rcv_data":82218671164040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411314249,"port_rcv_data":2269661399388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:13.966","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.3,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:13.985","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:14.003","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":124.25,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:14.019","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:14.035","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:14.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:14.068","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.53,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:14.082","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259034.864203,"monotonic":2345872.31327071,"hcas":{"mlx5_0":{"port_xmit_data":82836589408432,"port_rcv_data":84220468405299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895347190,"port_rcv_data":3196930483667,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82690241226970,"port_rcv_data":82218679621403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411314249,"port_rcv_data":2269661399388,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:14.968","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":132.85,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:14.987","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":123.1,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:15.022","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:15.038","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":127.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:15.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.8,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:15.068","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.31,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:15.082","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259035.8649628,"monotonic":2345873.314030582,"hcas":{"mlx5_0":{"port_xmit_data":82836599248372,"port_rcv_data":84221499553873,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895370874,"port_rcv_data":3196954416467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82691250971233,"port_rcv_data":82218713522736,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411337716,"port_rcv_data":2269685332188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:15.948","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:15.967","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:15.985","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.2,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:16.001","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.69,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:16.016","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:16.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:16.046","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.93,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:16.060","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.87,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259036.8657472,"monotonic":2345874.314814913,"hcas":{"mlx5_0":{"port_xmit_data":82836608439251,"port_rcv_data":84222450879913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895384871,"port_rcv_data":3196968776147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82692185158155,"port_rcv_data":82218737200592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411351898,"port_rcv_data":2269699691868,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:16.990","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.011","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.48,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:17.046","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":125.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:17.062","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:17.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:17.096","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.59,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:17.112","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259037.866475,"monotonic":2345875.315542154,"hcas":{"mlx5_0":{"port_xmit_data":82836617621120,"port_rcv_data":84223401409275,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895398867,"port_rcv_data":3196983135827,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82693122063091,"port_rcv_data":82218760892104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411365910,"port_rcv_data":2269714051548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:17.949","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:17.968","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:17.985","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.99,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:18.002","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:18.017","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:18.033","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:18.047","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:18.061","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259038.8672533,"monotonic":2345876.316321043,"hcas":{"mlx5_0":{"port_xmit_data":82836626619073,"port_rcv_data":84224333578844,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895413468,"port_rcv_data":3196997495507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82694039404811,"port_rcv_data":82218784416636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411380000,"port_rcv_data":2269728411228,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:18.972","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:18.992","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.86,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.009","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.22,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:19.025","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:19.041","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":133.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:19.056","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:19.071","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:19.085","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.37,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259039.8679929,"monotonic":2345877.317060793,"hcas":{"mlx5_0":{"port_xmit_data":82836635778686,"port_rcv_data":84225291034061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895437927,"port_rcv_data":3197022624947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82694973263692,"port_rcv_data":82218822453958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411407993,"port_rcv_data":2269757130588,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:19.953","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:19.972","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:19.990","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.32,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:20.007","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.35,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:20.023","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:20.038","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:20.056","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.77,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:20.071","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259040.8687532,"monotonic":2345878.317821137,"hcas":{"mlx5_0":{"port_xmit_data":82836645218601,"port_rcv_data":84226224492694,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895441446,"port_rcv_data":3197026214867,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82695903923305,"port_rcv_data":82218832048448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411407993,"port_rcv_data":2269757130588,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:20.985","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.004","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.82,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:21.022","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.02,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:21.039","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:21.054","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:21.073","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:21.088","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.66,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:21.103","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259041.8694909,"monotonic":2345879.31855897,"hcas":{"mlx5_0":{"port_xmit_data":82836655576075,"port_rcv_data":84227192362721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438895622889,"port_rcv_data":3197048816069,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82696850045631,"port_rcv_data":82218884921125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555411875535,"port_rcv_data":2269800537975,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:21.968","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:21.988","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.1,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:22.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.7,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:22.021","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:22.036","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:22.052","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:22.066","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:22.080","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259042.8702533,"monotonic":2345880.319321299,"hcas":{"mlx5_0":{"port_xmit_data":82836666023629,"port_rcv_data":84228234707620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438896034352,"port_rcv_data":3197160594039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82697781466933,"port_rcv_data":82218995837029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555412188743,"port_rcv_data":2269900351232,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:22.974","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.06,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:22.993","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:23.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.19,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:23.027","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:23.043","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:23.059","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:23.073","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:23.088","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259043.8710046,"monotonic":2345881.320072456,"hcas":{"mlx5_0":{"port_xmit_data":82836677242680,"port_rcv_data":84229170496445,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438896357015,"port_rcv_data":3197192498998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82698683572130,"port_rcv_data":82219029860264,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555412358825,"port_rcv_data":2269923414298,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:23.972","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.1,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:23.991","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:24.009","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.05,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:24.026","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.82,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:24.041","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:24.057","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:24.071","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:24.085","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259044.87175,"monotonic":2345882.320817961,"hcas":{"mlx5_0":{"port_xmit_data":82836688254885,"port_rcv_data":84230068770820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438896589965,"port_rcv_data":3197221702107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82699553648625,"port_rcv_data":82219079409537,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555412686014,"port_rcv_data":2269962360521,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:24.973","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.68,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:24.987","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:25.002","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.68,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:25.016","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:25.029","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:25.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:25.056","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.58,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:25.069","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259045.8725164,"monotonic":2345883.321583695,"hcas":{"mlx5_0":{"port_xmit_data":82836699452160,"port_rcv_data":84230990317617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438896862749,"port_rcv_data":3197261689075,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82700434904075,"port_rcv_data":82219120968569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555412859025,"port_rcv_data":2269992603496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:25.973","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:25.992","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.009","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.71,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:26.025","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:26.041","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:26.057","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:26.071","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":137.85,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:26.085","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.84,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259046.8732681,"monotonic":2345884.322336356,"hcas":{"mlx5_0":{"port_xmit_data":82836710072594,"port_rcv_data":84231992844556,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438896876730,"port_rcv_data":3197276048755,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82701425093704,"port_rcv_data":82219146040786,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555412872991,"port_rcv_data":2270006963176,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:26.970","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:26.984","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:26.997","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.73,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.009","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:27.020","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:27.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:27.041","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.38,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:27.051","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.22,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259047.8740723,"monotonic":2345885.323140366,"hcas":{"mlx5_0":{"port_xmit_data":82836721187222,"port_rcv_data":84233090372806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438896890975,"port_rcv_data":3197290408435,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82702508406484,"port_rcv_data":82219171528932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555412887018,"port_rcv_data":2270021322856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:27.954","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.84,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:27.968","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.85,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:27.981","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.13,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:27.994","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:28.006","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:28.017","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:28.027","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.61,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:28.037","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.47,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259048.8748806,"monotonic":2345886.323948642,"hcas":{"mlx5_0":{"port_xmit_data":82836732253441,"port_rcv_data":84234170937178,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438896890975,"port_rcv_data":3197290408435,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82703589412708,"port_rcv_data":82219200306109,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555413245921,"port_rcv_data":2270039006056,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:28.972","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.01,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:28.986","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:28.999","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.73,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.011","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:29.022","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:29.033","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:29.044","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.03,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:29.054","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.46,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259049.8756835,"monotonic":2345887.32475162,"hcas":{"mlx5_0":{"port_xmit_data":82836743881838,"port_rcv_data":84235277345208,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438897264261,"port_rcv_data":3197347446803,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82704639053995,"port_rcv_data":82219259870278,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555413459340,"port_rcv_data":2270087202696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:29.956","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:29.970","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:29.983","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.35,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:29.995","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:30.007","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:30.017","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:30.028","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:30.037","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259050.876502,"monotonic":2345888.325570081,"hcas":{"mlx5_0":{"port_xmit_data":82836755619587,"port_rcv_data":84236305576123,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438897635765,"port_rcv_data":3197418525845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82705596094058,"port_rcv_data":82219333508714,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555413674620,"port_rcv_data":2270149440138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:30.957","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:30.971","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:30.984","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.39,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:30.997","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:31.008","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.94,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:31.019","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:31.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.81,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:31.039","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259051.8773046,"monotonic":2345889.326372532,"hcas":{"mlx5_0":{"port_xmit_data":82836767031414,"port_rcv_data":84237290756422,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438897665773,"port_rcv_data":3197447637363,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82706550533385,"port_rcv_data":82219373978093,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555413704541,"port_rcv_data":2270178551656,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:31.958","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:31.972","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:31.985","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.1,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:31.998","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:32.009","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:32.021","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:32.031","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:32.041","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.77,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259052.8781095,"monotonic":2345890.327177539,"hcas":{"mlx5_0":{"port_xmit_data":82836778495846,"port_rcv_data":84238284015304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438897695967,"port_rcv_data":3197476708247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82707516728080,"port_rcv_data":82219414593141,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555413734533,"port_rcv_data":2270207622540,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:32.978","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:32.992","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:33.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.53,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:33.017","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:33.029","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:33.039","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:33.049","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:33.060","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259053.8789084,"monotonic":2345891.327976274,"hcas":{"mlx5_0":{"port_xmit_data":82836790049537,"port_rcv_data":84239268325684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438897863522,"port_rcv_data":3197503154508,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82708472994377,"port_rcv_data":82219468514022,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555414068698,"port_rcv_data":2270250172122,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:33.979","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:33.993","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.006","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.88,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.019","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:34.030","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:34.041","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:34.051","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.84,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:34.061","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259054.8797042,"monotonic":2345892.328772245,"hcas":{"mlx5_0":{"port_xmit_data":82836801778499,"port_rcv_data":84240271095362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438898192339,"port_rcv_data":3197545898324,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82709433238433,"port_rcv_data":82219506608212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555414244561,"port_rcv_data":2270276811738,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:34.960","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:34.974","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:34.987","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.11,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:34.999","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:35.010","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:35.021","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:35.031","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.59,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:35.041","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259055.8805096,"monotonic":2345893.329577806,"hcas":{"mlx5_0":{"port_xmit_data":82836813275028,"port_rcv_data":84241242138943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438898361165,"port_rcv_data":3197554602003,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82710397844602,"port_rcv_data":82219526701858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555414414441,"port_rcv_data":2270285515124,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:35.978","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:35.992","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.005","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.5,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:36.018","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:36.029","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:36.040","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":142.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:36.050","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.99,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:36.060","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.78,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259056.8812993,"monotonic":2345894.330367368,"hcas":{"mlx5_0":{"port_xmit_data":82836824533670,"port_rcv_data":84242343879241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438898529046,"port_rcv_data":3197573841835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82711478796463,"port_rcv_data":82219574780947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555414942417,"port_rcv_data":2270322438156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:36.962","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:36.976","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:36.989","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.58,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:37.002","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:37.014","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:37.025","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:37.035","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:37.045","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259057.8821166,"monotonic":2345895.331184534,"hcas":{"mlx5_0":{"port_xmit_data":82836835930259,"port_rcv_data":84243465389548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438898866698,"port_rcv_data":3197616204663,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82712560363983,"port_rcv_data":82219619357637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555415123411,"port_rcv_data":2270355958084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:37.982","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:37.996","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:38.010","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.53,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:38.022","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:38.033","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:38.044","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:38.055","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.62,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:38.065","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259058.8829124,"monotonic":2345896.331980279,"hcas":{"mlx5_0":{"port_xmit_data":82836847795756,"port_rcv_data":84244436471484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438899082582,"port_rcv_data":3197683996609,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82713462940769,"port_rcv_data":82219690043568,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555415183473,"port_rcv_data":2270414909602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:38.983","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:38.998","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.011","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.22,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:39.023","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:39.035","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:39.046","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:39.056","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.87,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:39.067","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259059.8837082,"monotonic":2345897.3327762,"hcas":{"mlx5_0":{"port_xmit_data":82836861289192,"port_rcv_data":84245428463031,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438899112745,"port_rcv_data":3197713532837,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82714426295428,"port_rcv_data":82219733092295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555415213233,"port_rcv_data":2270444445830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:39.965","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:39.979","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:39.992","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.15,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:40.004","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:40.015","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:40.026","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:40.036","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.41,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:40.045","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259060.884512,"monotonic":2345898.333580036,"hcas":{"mlx5_0":{"port_xmit_data":82836874716079,"port_rcv_data":84246413905600,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438899142861,"port_rcv_data":3197743076467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82715382322595,"port_rcv_data":82219776061380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555415243179,"port_rcv_data":2270473989460,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:40.966","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:40.980","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.33,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:40.993","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.74,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:41.006","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:41.017","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:41.028","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:41.039","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:41.049","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259061.8853269,"monotonic":2345899.334394951,"hcas":{"mlx5_0":{"port_xmit_data":82836888154843,"port_rcv_data":84247402058603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438899173117,"port_rcv_data":3197772612695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82716338835090,"port_rcv_data":82219819026352,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555415273110,"port_rcv_data":2270503525688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:41.966","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:41.980","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:41.994","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.58,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:42.007","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:42.019","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:42.030","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:42.040","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.72,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:42.051","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259062.8861427,"monotonic":2345900.335210755,"hcas":{"mlx5_0":{"port_xmit_data":82836901513439,"port_rcv_data":84248369129162,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438899187641,"port_rcv_data":3197786973021,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82717292016691,"port_rcv_data":82219873705686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555415806139,"port_rcv_data":2270544824207,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:42.987","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:43.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.14,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:43.014","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.03,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:43.026","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:43.037","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:43.048","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:43.058","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:43.068","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.27,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259063.8869448,"monotonic":2345901.336012595,"hcas":{"mlx5_0":{"port_xmit_data":82836915306211,"port_rcv_data":84249347377143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438899660747,"port_rcv_data":3197812526604,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82718244937257,"port_rcv_data":82219896058045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555415977600,"port_rcv_data":2270553803958,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:43.986","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.013","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.31,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:44.025","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:44.036","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:44.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:44.057","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.79,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:44.066","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.95,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259064.8877332,"monotonic":2345902.33680111,"hcas":{"mlx5_0":{"port_xmit_data":82836928632272,"port_rcv_data":84250402500010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438899948861,"port_rcv_data":3197900761149,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82719214151701,"port_rcv_data":82220004774357,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555416371502,"port_rcv_data":2270649353698,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:44.969","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:44.983","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:44.996","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.84,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:45.008","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:45.019","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.44,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:45.030","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:45.040","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.63,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:45.050","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259065.8885407,"monotonic":2345903.337608746,"hcas":{"mlx5_0":{"port_xmit_data":82836941281263,"port_rcv_data":84251490338159,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438900136473,"port_rcv_data":3197938573569,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82720262606129,"port_rcv_data":82220063916628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555416712936,"port_rcv_data":2270696006546,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.003","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.59,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:46.017","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:46.030","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.85,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:46.043","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:46.055","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:46.066","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:46.077","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:46.088","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":145.08,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259066.8893182,"monotonic":2345904.338386089,"hcas":{"mlx5_0":{"port_xmit_data":82836954577506,"port_rcv_data":84252463619192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438900645664,"port_rcv_data":3198003658595,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82721172927369,"port_rcv_data":82220115336216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555416752259,"port_rcv_data":2270734568072,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:46.994","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:47.009","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:47.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.37,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:47.037","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:47.050","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:47.064","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:47.077","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:47.091","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259067.8900938,"monotonic":2345905.339161555,"hcas":{"mlx5_0":{"port_xmit_data":82836967345389,"port_rcv_data":84253393505137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438900701758,"port_rcv_data":3198061097961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82722045325414,"port_rcv_data":82220211447092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555417259047,"port_rcv_data":2270817909497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:47.993","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:48.012","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:48.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.86,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:48.045","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:48.061","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:48.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:48.091","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.38,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:48.105","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.92,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259068.8908367,"monotonic":2345906.33990462,"hcas":{"mlx5_0":{"port_xmit_data":82836980336666,"port_rcv_data":84254307448348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438901034404,"port_rcv_data":3198107086300,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82722912112905,"port_rcv_data":82220261494403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555417434399,"port_rcv_data":2270855263230,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:48.993","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:49.013","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:49.030","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.67,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:49.047","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:49.063","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:49.078","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:49.093","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.76,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:49.107","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259069.8915887,"monotonic":2345907.340656564,"hcas":{"mlx5_0":{"port_xmit_data":82836993220893,"port_rcv_data":84255214764020,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438901363484,"port_rcv_data":3198153074423,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82723774417477,"port_rcv_data":82220302858498,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555417462454,"port_rcv_data":2270883982728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:49.992","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:50.011","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:50.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.91,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:50.050","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:50.068","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:50.085","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:50.101","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.11,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:50.116","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259070.8923264,"monotonic":2345908.341394168,"hcas":{"mlx5_0":{"port_xmit_data":82837005438543,"port_rcv_data":84256120433184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438901377465,"port_rcv_data":3198167434103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82724666712761,"port_rcv_data":82220338343693,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555417663287,"port_rcv_data":2270907184008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:50.992","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.011","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:51.029","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.86,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:51.045","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:51.061","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:51.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:51.090","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:51.104","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.88,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259071.8930829,"monotonic":2345909.342150826,"hcas":{"mlx5_0":{"port_xmit_data":82837017326758,"port_rcv_data":84257088621576,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438901759850,"port_rcv_data":3198209875215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82725591076996,"port_rcv_data":82220383585784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555417858990,"port_rcv_data":2270940783520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:51.976","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:51.994","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:52.012","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.5,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:52.028","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:52.044","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:52.059","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:52.074","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.67,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:52.088","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259072.8938487,"monotonic":2345910.342916446,"hcas":{"mlx5_0":{"port_xmit_data":82837029340712,"port_rcv_data":84258039996655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438902128580,"port_rcv_data":3198257711959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82726496502819,"port_rcv_data":82220443197947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555418247436,"port_rcv_data":2270988620136,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.033","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:53.052","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.33,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:53.070","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.23,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:53.086","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:53.102","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:53.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:53.131","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.29,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:53.145","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259073.8945646,"monotonic":2345911.343632322,"hcas":{"mlx5_0":{"port_xmit_data":82837042260753,"port_rcv_data":84258946931996,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438902317525,"port_rcv_data":3198292044647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82727369932647,"port_rcv_data":82220499261450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555418597038,"port_rcv_data":2271031793804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:53.997","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:54.016","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:54.034","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.46,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:54.050","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:54.066","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:54.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:54.095","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.45,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:54.109","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.38,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259074.895315,"monotonic":2345912.344382951,"hcas":{"mlx5_0":{"port_xmit_data":82837055292851,"port_rcv_data":84259887433788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438902686053,"port_rcv_data":3198364406631,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82728236406349,"port_rcv_data":82220575467799,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555418816983,"port_rcv_data":2271095314188,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:54.997","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:55.015","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:55.032","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.56,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:55.048","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:55.062","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.98,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:55.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:55.090","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.68,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:55.104","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259075.8960736,"monotonic":2345913.34514138,"hcas":{"mlx5_0":{"port_xmit_data":82837068230699,"port_rcv_data":84260841836710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438902917034,"port_rcv_data":3198447425027,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82729109741554,"port_rcv_data":82220671269416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555419049623,"port_rcv_data":2271178274841,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:55.996","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:56.015","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.41,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:56.033","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.49,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:56.049","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:56.065","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:56.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:56.095","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:56.109","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259076.8968174,"monotonic":2345914.345885216,"hcas":{"mlx5_0":{"port_xmit_data":82837080768472,"port_rcv_data":84261732444786,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438902947197,"port_rcv_data":3198476824118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82729971005272,"port_rcv_data":82220722223287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555419239172,"port_rcv_data":2271216711073,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.018","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:57.037","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:57.054","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.4,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:57.071","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:57.087","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:57.102","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:57.117","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:57.131","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.31,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259077.8975503,"monotonic":2345915.346618188,"hcas":{"mlx5_0":{"port_xmit_data":82837093867659,"port_rcv_data":84262650254863,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438903292800,"port_rcv_data":3198524240471,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82730840486756,"port_rcv_data":82220773445054,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555419430148,"port_rcv_data":2271255089699,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:57.980","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:58.000","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:58.017","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.27,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:58.033","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:58.049","gpu_util_pct":99.0,"memory_used_mib":79875.0,"power_w":142.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:58.064","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:58.078","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.3,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:58.092","gpu_util_pct":99.0,"memory_used_mib":80005.0,"power_w":141.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259078.8983316,"monotonic":2345916.347399464,"hcas":{"mlx5_0":{"port_xmit_data":82837107503418,"port_rcv_data":84263427739039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438903491355,"port_rcv_data":3198565151283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82731578666805,"port_rcv_data":82220834318650,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555419777999,"port_rcv_data":2271302505613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:37:59.019","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:37:59.037","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:37:59.055","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.97,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:37:59.071","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:37:59.086","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:37:59.101","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:37:59.116","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.22,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:37:59.130","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259079.8990712,"monotonic":2345917.348139124,"hcas":{"mlx5_0":{"port_xmit_data":82837121051470,"port_rcv_data":84264268198574,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438903809958,"port_rcv_data":3198594878186,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82732390057892,"port_rcv_data":82220879539387,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555420152138,"port_rcv_data":2271334496038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.017","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:00.036","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:00.055","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.93,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:00.071","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:00.087","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:00.102","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:00.117","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.31,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:00.131","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259080.8998306,"monotonic":2345918.348897959,"hcas":{"mlx5_0":{"port_xmit_data":82837134099084,"port_rcv_data":84265155788721,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438904158475,"port_rcv_data":3198612716604,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82733258240848,"port_rcv_data":82220901376497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555420327334,"port_rcv_data":2271343622110,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:00.984","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:01.003","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:01.021","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.8,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:01.038","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.28,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:01.054","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:01.069","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:01.084","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.54,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:01.098","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259081.9005964,"monotonic":2345919.349664331,"hcas":{"mlx5_0":{"port_xmit_data":82837147074308,"port_rcv_data":84266090908995,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438904365586,"port_rcv_data":3198671073353,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82734137290853,"port_rcv_data":82220981433889,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555420700202,"port_rcv_data":2271410818866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:02.002","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:02.021","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:02.039","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.44,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:02.055","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:02.071","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:02.089","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:02.104","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":135.81,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:02.118","gpu_util_pct":98.0,"memory_used_mib":80005.0,"power_w":131.82,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259082.901354,"monotonic":2345920.350421948,"hcas":{"mlx5_0":{"port_xmit_data":82837158626983,"port_rcv_data":84267027290186,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438904550532,"port_rcv_data":3198709321669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82735035674232,"port_rcv_data":82221048893598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555421197442,"port_rcv_data":2271466751554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:03.000","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:03.020","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:03.037","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.05,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:03.054","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:03.069","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:03.084","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:03.099","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:03.113","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259083.9021175,"monotonic":2345921.35118534,"hcas":{"mlx5_0":{"port_xmit_data":82837170605965,"port_rcv_data":84268006854977,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438905051942,"port_rcv_data":3198765390099,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82735959274117,"port_rcv_data":82221090001647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555421227807,"port_rcv_data":2271496295184,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:04.005","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:04.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:04.042","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.36,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:04.058","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:04.074","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:04.089","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:04.104","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:04.118","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.41,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259084.9028764,"monotonic":2345922.351944303,"hcas":{"mlx5_0":{"port_xmit_data":82837181879499,"port_rcv_data":84268938869973,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438905082508,"port_rcv_data":3198794926327,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82736861738523,"port_rcv_data":82221130918336,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555421258249,"port_rcv_data":2271525831412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.009","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:05.033","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:05.050","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.91,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:05.066","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:05.082","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:05.096","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:05.111","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:05.124","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259085.9036376,"monotonic":2345923.352704683,"hcas":{"mlx5_0":{"port_xmit_data":82837193372561,"port_rcv_data":84269902000337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438905128248,"port_rcv_data":3198839421745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82737783209207,"port_rcv_data":82221187186519,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555421304206,"port_rcv_data":2271570551590,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:05.987","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:06.006","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:06.023","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.59,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:06.040","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:06.056","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:06.071","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:06.086","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:06.099","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259086.9044282,"monotonic":2345924.353495408,"hcas":{"mlx5_0":{"port_xmit_data":82837205728854,"port_rcv_data":84270794863244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438905152583,"port_rcv_data":3198863579305,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82738648939147,"port_rcv_data":82221223516646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555421327658,"port_rcv_data":2271594484390,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:07.004","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:07.023","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:07.040","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.9,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:07.057","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:07.072","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:07.087","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:07.101","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:07.116","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259087.9051974,"monotonic":2345925.354265076,"hcas":{"mlx5_0":{"port_xmit_data":82837218457993,"port_rcv_data":84271677373309,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438905163960,"port_rcv_data":3198873887089,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82739522587802,"port_rcv_data":82221290213815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555421949409,"port_rcv_data":2271648386678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:08.027","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:08.045","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:08.063","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.26,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:08.079","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:08.094","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:08.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:08.124","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:08.137","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.48,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259088.9059453,"monotonic":2345926.355013037,"hcas":{"mlx5_0":{"port_xmit_data":82837231794604,"port_rcv_data":84272626874633,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438905679103,"port_rcv_data":3198944290806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82740400896207,"port_rcv_data":82221339165394,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555422091374,"port_rcv_data":2271684405878,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:09.008","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:09.028","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:09.045","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.47,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:09.061","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:09.077","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:09.092","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:09.106","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:09.120","gpu_util_pct":99.0,"memory_used_mib":80005.0,"power_w":137.16,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259089.9067097,"monotonic":2345927.355777678,"hcas":{"mlx5_0":{"port_xmit_data":82837244541683,"port_rcv_data":84273545853106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438905870202,"port_rcv_data":3198982814499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82741280878835,"port_rcv_data":82221398794252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555422435939,"port_rcv_data":2271731400992,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:10.008","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:10.027","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:10.045","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.79,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:10.061","gpu_util_pct":98.0,"memory_used_mib":79963.0,"power_w":130.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:10.077","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:10.092","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":141.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:10.106","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:10.120","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.55,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259090.90748,"monotonic":2345928.356547868,"hcas":{"mlx5_0":{"port_xmit_data":82837256624119,"port_rcv_data":84274538550606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438906216147,"port_rcv_data":3199029954317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82742228656679,"port_rcv_data":82221448950322,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555422627581,"port_rcv_data":2271769641467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.025","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:11.044","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:11.062","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.2,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:11.078","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.37,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:11.093","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:11.108","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:11.122","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:11.136","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259091.9082336,"monotonic":2345929.357301182,"hcas":{"mlx5_0":{"port_xmit_data":82837268826210,"port_rcv_data":84275541805894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438906556682,"port_rcv_data":3199077036393,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82743185480465,"port_rcv_data":82221499135858,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555422814278,"port_rcv_data":2271807939686,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:11.991","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:12.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:12.028","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.82,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:12.044","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:12.059","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:12.074","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.27,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:12.090","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:12.104","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259092.9090202,"monotonic":2345930.358087716,"hcas":{"mlx5_0":{"port_xmit_data":82837280225202,"port_rcv_data":84276484108079,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438906587279,"port_rcv_data":3199106464140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82744097067059,"port_rcv_data":82221540026321,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555422844999,"port_rcv_data":2271837367433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:12.991","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:13.010","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:13.028","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.19,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:13.044","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.98,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:13.059","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:13.074","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:13.088","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:13.102","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.32,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259093.9098058,"monotonic":2345931.358873392,"hcas":{"mlx5_0":{"port_xmit_data":82837291615549,"port_rcv_data":84277412683933,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438906603073,"port_rcv_data":3199121503335,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82745012361706,"port_rcv_data":82221593082315,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555423400442,"port_rcv_data":2271878989171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:14.012","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:14.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:14.048","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.98,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:14.064","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:14.080","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:14.095","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:14.109","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.92,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:14.123","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259094.9105763,"monotonic":2345932.359644034,"hcas":{"mlx5_0":{"port_xmit_data":82837303379517,"port_rcv_data":84278381169475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438906961774,"port_rcv_data":3199174031351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82745928295058,"port_rcv_data":82221648365806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555423601833,"port_rcv_data":2271922674967,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.012","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:15.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:15.048","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.76,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:15.065","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.76,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:15.080","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:15.095","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:15.110","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:15.124","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.64,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259095.910964,"monotonic":2345933.360031595,"hcas":{"mlx5_0":{"port_xmit_data":82837315470389,"port_rcv_data":84279400380330,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438907343121,"port_rcv_data":3199245680192,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82746876229564,"port_rcv_data":82221723050822,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555423812943,"port_rcv_data":2271985559239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:15.994","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:16.012","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:16.030","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.39,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:16.046","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:16.061","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:16.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:16.090","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:16.104","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259096.9117515,"monotonic":2345934.360819104,"hcas":{"mlx5_0":{"port_xmit_data":82837327601514,"port_rcv_data":84280412216907,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438907707262,"port_rcv_data":3199314278643,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82747819382494,"port_rcv_data":82221821603423,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555424190709,"port_rcv_data":2272072313869,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:17.029","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:17.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:17.065","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.53,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:17.082","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:17.097","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:17.112","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:17.127","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:17.141","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259097.9125028,"monotonic":2345935.361570209,"hcas":{"mlx5_0":{"port_xmit_data":82837339687027,"port_rcv_data":84281438314883,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438908087896,"port_rcv_data":3199397651026,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82748763025225,"port_rcv_data":82221889687848,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555424394875,"port_rcv_data":2272128610270,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:18.013","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:18.032","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:18.049","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.28,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:18.066","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:18.081","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.18,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:18.096","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:18.111","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:18.125","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259098.913272,"monotonic":2345936.362339485,"hcas":{"mlx5_0":{"port_xmit_data":82837350618463,"port_rcv_data":84282426425127,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438908132210,"port_rcv_data":3199441438453,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82749705603967,"port_rcv_data":82221944451838,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555424438942,"port_rcv_data":2272172397697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:19.016","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:19.034","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.69,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:19.052","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.98,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:19.068","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:19.084","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:19.099","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:19.114","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:19.127","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259099.9140375,"monotonic":2345937.363105133,"hcas":{"mlx5_0":{"port_xmit_data":82837361439244,"port_rcv_data":84283456863186,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438908160606,"port_rcv_data":3199470158244,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82750707729648,"port_rcv_data":82222001908391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555424828348,"port_rcv_data":2272218954751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.033","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:20.054","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:20.072","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.77,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:20.088","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:20.104","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":133.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:20.119","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:20.134","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:20.147","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259100.9147892,"monotonic":2345938.363856831,"hcas":{"mlx5_0":{"port_xmit_data":82837372321035,"port_rcv_data":84284451297023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438908353395,"port_rcv_data":3199493437349,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82751681510238,"port_rcv_data":82222044919067,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555425179563,"port_rcv_data":2272251152194,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:20.997","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:21.016","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:21.034","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.06,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:21.050","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:21.066","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:21.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:21.095","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.95,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:21.109","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.6,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259101.9155777,"monotonic":2345939.364645359,"hcas":{"mlx5_0":{"port_xmit_data":82837384294816,"port_rcv_data":84285399836825,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438908868336,"port_rcv_data":3199520194329,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82752603676876,"port_rcv_data":82222059679817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555425215941,"port_rcv_data":2272254378601,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:22.016","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:22.035","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:22.052","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.7,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:22.068","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:22.084","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:22.099","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:22.114","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:22.128","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259102.9163482,"monotonic":2345940.365415978,"hcas":{"mlx5_0":{"port_xmit_data":82837395558593,"port_rcv_data":84286411407718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438909072626,"port_rcv_data":3199568183666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82753565521738,"port_rcv_data":82222133442074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555425706408,"port_rcv_data":2272316857137,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:23.033","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:23.052","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:23.070","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.36,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:23.086","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.73,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:23.101","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:23.116","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:23.132","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:23.146","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259103.9171004,"monotonic":2345941.366168273,"hcas":{"mlx5_0":{"port_xmit_data":82837406628147,"port_rcv_data":84287448297007,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438909409627,"port_rcv_data":3199609799666,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82754563299567,"port_rcv_data":82222177118256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555425883759,"port_rcv_data":2272349631537,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:24.037","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:24.056","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:24.074","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.0,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:24.090","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:24.106","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.89,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:24.121","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:24.135","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:24.149","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.29,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259104.9178486,"monotonic":2345942.366916432,"hcas":{"mlx5_0":{"port_xmit_data":82837417388810,"port_rcv_data":84288452465066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438909582685,"port_rcv_data":3199633000946,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82755544415201,"port_rcv_data":82222219892750,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555426216544,"port_rcv_data":2272381674417,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:25.020","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.72,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:25.038","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:25.055","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.89,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:25.071","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.33,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:25.086","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:25.101","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:25.116","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:25.129","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259105.9186296,"monotonic":2345943.367697278,"hcas":{"mlx5_0":{"port_xmit_data":82837428527169,"port_rcv_data":84289485493617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438909907673,"port_rcv_data":3199665043826,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82756543925969,"port_rcv_data":82222260333984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555426454949,"port_rcv_data":2272411285857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:26.001","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:26.020","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:26.037","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.18,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:26.053","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:26.068","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:26.083","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:26.098","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:26.112","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.56,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259106.9194221,"monotonic":2345944.368489773,"hcas":{"mlx5_0":{"port_xmit_data":82837439827449,"port_rcv_data":84290547013741,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438910247635,"port_rcv_data":3199711446817,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82757559432545,"port_rcv_data":82222312116591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555426733562,"port_rcv_data":2272452018591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:27.020","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:27.038","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:27.056","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.59,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:27.072","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:27.087","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:27.102","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:27.116","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:27.131","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.79,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259107.9201925,"monotonic":2345945.36926015,"hcas":{"mlx5_0":{"port_xmit_data":82837450675273,"port_rcv_data":84291564280880,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438910472106,"port_rcv_data":3199738859542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82758551772311,"port_rcv_data":82222340703829,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555426801700,"port_rcv_data":2272469848641,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:28.038","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":142.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:28.057","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":141.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:28.074","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.56,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:28.090","gpu_util_pct":98.0,"memory_used_mib":79963.0,"power_w":130.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:28.106","gpu_util_pct":98.0,"memory_used_mib":79875.0,"power_w":136.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:28.120","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":140.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:28.135","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:28.148","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259108.9209478,"monotonic":2345946.370014942,"hcas":{"mlx5_0":{"port_xmit_data":82837460565942,"port_rcv_data":84292576594824,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438910472106,"port_rcv_data":3199738859542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82759561614432,"port_rcv_data":82222359555691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555426982213,"port_rcv_data":2272478690241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:29.044","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:29.063","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:29.080","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.32,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:29.096","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:29.112","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":133.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:29.126","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:29.141","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:29.156","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259109.9217083,"monotonic":2345947.370775789,"hcas":{"mlx5_0":{"port_xmit_data":82837470113045,"port_rcv_data":84293624029642,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438910901595,"port_rcv_data":3199853336628,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82760497857344,"port_rcv_data":82222492256017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555427606661,"port_rcv_data":2272602008307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:30.024","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:30.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:30.060","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.69,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:30.076","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":125.89,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:30.092","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":133.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:30.107","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:30.123","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:30.136","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.98,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259110.9224813,"monotonic":2345948.37154922,"hcas":{"mlx5_0":{"port_xmit_data":82837480019731,"port_rcv_data":84294619883367,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911220058,"port_rcv_data":3199871020474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82761476969066,"port_rcv_data":82222527715061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428044149,"port_rcv_data":2272627691328,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:31.025","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:31.045","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:31.063","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.29,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:31.079","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:31.095","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:31.110","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:31.125","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:31.139","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259111.9232392,"monotonic":2345949.372306705,"hcas":{"mlx5_0":{"port_xmit_data":82837490139589,"port_rcv_data":84295627515264,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911813677,"port_rcv_data":3199905265304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82762448418778,"port_rcv_data":82222545959415,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428188082,"port_rcv_data":2272636252611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:32.063","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:32.082","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:32.099","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.21,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:32.115","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.0,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:32.131","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:32.146","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:32.160","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:32.174","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.51,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259112.9239504,"monotonic":2345950.37301828,"hcas":{"mlx5_0":{"port_xmit_data":82837499944421,"port_rcv_data":84296631863087,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911813677,"port_rcv_data":3199905265304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82763454562689,"port_rcv_data":82222555940675,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428188082,"port_rcv_data":2272636252611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:33.007","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:33.026","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:33.043","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.11,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:33.060","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:33.075","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":133.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:33.093","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:33.108","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:33.122","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259113.9247205,"monotonic":2345951.373788525,"hcas":{"mlx5_0":{"port_xmit_data":82837507831776,"port_rcv_data":84297482528161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911813677,"port_rcv_data":3199905265304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82764305440137,"port_rcv_data":82222564068288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428188082,"port_rcv_data":2272636252611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:34.026","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":131.35,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:34.045","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":129.6,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:34.062","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":121.58,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:34.079","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:34.095","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:34.110","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":130.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:34.125","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.65,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:34.141","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":124.23,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259114.9254715,"monotonic":2345952.374539397,"hcas":{"mlx5_0":{"port_xmit_data":82837515790775,"port_rcv_data":84298368181082,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911813677,"port_rcv_data":3199905265304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82765191163071,"port_rcv_data":82222572283951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428188082,"port_rcv_data":2272636252611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:35.008","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:35.026","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:35.044","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":121.67,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:35.061","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":124.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:35.076","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":121.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:35.092","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:35.106","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.98,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:35.122","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":125.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259115.926243,"monotonic":2345953.375310886,"hcas":{"mlx5_0":{"port_xmit_data":82837523696889,"port_rcv_data":84299237020491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911813677,"port_rcv_data":3199905265304,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82766060266582,"port_rcv_data":82222580440288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428188082,"port_rcv_data":2272636252611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:36.028","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:36.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:36.065","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.65,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:36.081","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:36.096","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":126.21,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:36.111","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:36.127","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:36.142","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":121.63,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259116.9269986,"monotonic":2345954.376066337,"hcas":{"mlx5_0":{"port_xmit_data":82837533305266,"port_rcv_data":84300229311197,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911823101,"port_rcv_data":3199914838424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82767043087442,"port_rcv_data":82222599778727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428197521,"port_rcv_data":2272645825731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:37.048","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:37.066","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:37.083","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.44,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:37.099","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:37.115","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":133.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:37.130","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:37.145","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.52,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:37.159","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.06,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259117.9277294,"monotonic":2345955.376797246,"hcas":{"mlx5_0":{"port_xmit_data":82837543306357,"port_rcv_data":84301279755287,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911851265,"port_rcv_data":3199943557784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82768063575556,"port_rcv_data":82222638588636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428225809,"port_rcv_data":2272674545091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:38.046","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:38.065","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:38.083","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.91,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:38.100","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:38.115","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:38.130","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:38.145","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:38.159","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259118.9284635,"monotonic":2345956.37753121,"hcas":{"mlx5_0":{"port_xmit_data":82837552904057,"port_rcv_data":84302273455982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911865261,"port_rcv_data":3199957917464,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82769044416404,"port_rcv_data":82222662710217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428239836,"port_rcv_data":2272688904771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:39.050","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":138.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:39.068","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:39.086","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.55,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:39.102","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:39.118","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:39.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:39.147","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.71,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:39.161","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.67,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259119.9291885,"monotonic":2345957.378256679,"hcas":{"mlx5_0":{"port_xmit_data":82837561989658,"port_rcv_data":84303218828712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911879862,"port_rcv_data":3199972277144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82769975791137,"port_rcv_data":82222686341144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428253957,"port_rcv_data":2272703264451,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:40.029","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:40.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.01,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:40.065","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.14,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:40.081","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:40.097","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:40.113","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:40.128","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:40.142","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259120.9299312,"monotonic":2345958.378999185,"hcas":{"mlx5_0":{"port_xmit_data":82837571756877,"port_rcv_data":84304234911001,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911893859,"port_rcv_data":3199986636824,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82770976181336,"port_rcv_data":82222710579899,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428267969,"port_rcv_data":2272717624131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:41.027","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:41.041","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:41.054","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.36,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:41.067","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.06,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:41.079","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:41.090","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:41.101","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.37,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:41.111","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.59,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259121.9307156,"monotonic":2345959.37978355,"hcas":{"mlx5_0":{"port_xmit_data":82837582775552,"port_rcv_data":84305438662807,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911907855,"port_rcv_data":3200000996504,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82772167775415,"port_rcv_data":82222736003310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428281981,"port_rcv_data":2272731983811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:42.048","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:42.062","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:42.076","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.67,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:42.088","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:42.100","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:42.111","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:42.122","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:42.133","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259122.931493,"monotonic":2345960.380561284,"hcas":{"mlx5_0":{"port_xmit_data":82837593849983,"port_rcv_data":84306634418984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438911907855,"port_rcv_data":3200000996504,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82773363977019,"port_rcv_data":82222773627375,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555428811197,"port_rcv_data":2272758508611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:43.048","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:43.061","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:43.075","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.43,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:43.087","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.25,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:43.099","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:43.110","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:43.120","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:43.130","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259123.9322681,"monotonic":2345961.381335903,"hcas":{"mlx5_0":{"port_xmit_data":82837605537053,"port_rcv_data":84307959783412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438912488594,"port_rcv_data":3200126517450,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82774564139962,"port_rcv_data":82222892734292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429084292,"port_rcv_data":2272866345737,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:44.031","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:44.045","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:44.058","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.83,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:44.070","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:44.081","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:44.092","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:44.102","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:44.113","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.8,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259124.9330654,"monotonic":2345962.382133514,"hcas":{"mlx5_0":{"port_xmit_data":82837616790847,"port_rcv_data":84309166770849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438912644384,"port_rcv_data":3200135359696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82775761388499,"port_rcv_data":82222930114924,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429535419,"port_rcv_data":2272892712758,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:45.031","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:45.044","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.88,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:45.058","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.1,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:45.070","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:45.082","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:45.093","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:45.103","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:45.113","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259125.9338548,"monotonic":2345963.382922791,"hcas":{"mlx5_0":{"port_xmit_data":82837628378538,"port_rcv_data":84310393940552,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913106269,"port_rcv_data":3200161520139,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82776963813215,"port_rcv_data":82222950023676,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686730,"port_rcv_data":2272901501903,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:46.015","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:46.029","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:46.042","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.14,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:46.054","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:46.065","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:46.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:46.087","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:46.096","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.09,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259126.9346755,"monotonic":2345964.383743472,"hcas":{"mlx5_0":{"port_xmit_data":82837639429002,"port_rcv_data":84311592457658,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82778153296020,"port_rcv_data":82222960946256,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:47.034","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:47.048","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.24,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:47.061","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.6,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:47.074","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:47.085","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.45,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:47.096","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:47.106","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.74,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:47.116","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.22,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259127.9354625,"monotonic":2345965.384530334,"hcas":{"mlx5_0":{"port_xmit_data":82837650084147,"port_rcv_data":84312854285479,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82779416015986,"port_rcv_data":82222971614482,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:48.035","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:48.049","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:48.062","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.15,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:48.075","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:48.086","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:48.096","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:48.107","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.23,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:48.117","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.4,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259128.9362493,"monotonic":2345966.3853173,"hcas":{"mlx5_0":{"port_xmit_data":82837660553094,"port_rcv_data":84314092733495,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82780653585693,"port_rcv_data":82222982096579,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:49.017","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:49.031","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:49.045","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.38,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:49.057","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:49.069","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.82,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:49.080","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:49.091","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.47,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:49.101","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.62,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259129.9370565,"monotonic":2345967.386124422,"hcas":{"mlx5_0":{"port_xmit_data":82837671140625,"port_rcv_data":84315346565664,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82781907927251,"port_rcv_data":82222992702009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:50.034","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:50.048","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:50.062","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.64,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:50.075","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:50.087","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:50.098","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:50.108","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:50.118","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259130.9378548,"monotonic":2345968.386922616,"hcas":{"mlx5_0":{"port_xmit_data":82837681671524,"port_rcv_data":84316593783070,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82783152958675,"port_rcv_data":82223003213785,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:51.034","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:51.048","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:51.061","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.16,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:51.073","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:51.085","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":147.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:51.096","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:51.106","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:51.116","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.18,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259131.938653,"monotonic":2345969.387720696,"hcas":{"mlx5_0":{"port_xmit_data":82837692334264,"port_rcv_data":84317856302466,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82784417946486,"port_rcv_data":82223013911711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:52.038","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:52.052","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:52.065","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.59,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:52.077","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:52.089","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:52.100","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:52.110","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:52.120","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.02,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259132.9394531,"monotonic":2345970.388520921,"hcas":{"mlx5_0":{"port_xmit_data":82837702902518,"port_rcv_data":84319106375988,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82785667681955,"port_rcv_data":82223024492768,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:53.055","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:53.069","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.36,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:53.083","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.71,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:53.095","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:53.106","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:53.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:53.127","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:53.137","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.12,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259133.9402397,"monotonic":2345971.389307341,"hcas":{"mlx5_0":{"port_xmit_data":82837713574749,"port_rcv_data":84320369902076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82786930676508,"port_rcv_data":82223035168696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:54.039","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:54.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.37,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:54.067","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.51,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:54.079","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.04,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:54.091","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.52,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:54.102","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:54.112","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.45,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:54.122","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.75,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259134.9410322,"monotonic":2345972.390100053,"hcas":{"mlx5_0":{"port_xmit_data":82837724078122,"port_rcv_data":84321613569756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82788174623743,"port_rcv_data":82223045684040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:55.022","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:55.036","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:55.050","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.36,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:55.063","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.6,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:55.074","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.6,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:55.085","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:55.096","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:55.106","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.21,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259135.941845,"monotonic":2345973.390912814,"hcas":{"mlx5_0":{"port_xmit_data":82837734719298,"port_rcv_data":84322873819273,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82789433082010,"port_rcv_data":82223056319023,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:56.039","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:56.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.09,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:56.066","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.16,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:56.079","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.23,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:56.090","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:56.101","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:56.112","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:56.122","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.84,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259136.9426467,"monotonic":2345974.391714659,"hcas":{"mlx5_0":{"port_xmit_data":82837745230908,"port_rcv_data":84324117201284,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82790678465519,"port_rcv_data":82223066865784,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:57.042","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:57.056","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:57.068","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.66,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:57.080","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:57.091","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.97,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:57.102","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:57.112","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:57.122","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.03,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259137.943443,"monotonic":2345975.392510966,"hcas":{"mlx5_0":{"port_xmit_data":82837755710892,"port_rcv_data":84325354991684,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82791915258510,"port_rcv_data":82223077339808,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:58.022","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:58.036","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:58.050","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.86,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:58.062","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.99,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:58.073","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.04,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:58.084","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:58.094","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:58.104","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.78,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259138.9442613,"monotonic":2345976.393329423,"hcas":{"mlx5_0":{"port_xmit_data":82837766368219,"port_rcv_data":84326617054059,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82793178166718,"port_rcv_data":82223088011561,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:38:59.024","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:38:59.038","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:38:59.051","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.41,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:38:59.064","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.22,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:38:59.076","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:38:59.087","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:38:59.097","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.64,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:38:59.107","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":145.46,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259139.9450753,"monotonic":2345977.394143051,"hcas":{"mlx5_0":{"port_xmit_data":82837776974119,"port_rcv_data":84327873423926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82794434817870,"port_rcv_data":82223098633529,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:00.025","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:00.039","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:00.052","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.59,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:00.064","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:00.075","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:00.086","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:00.096","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.13,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:00.105","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259140.9459054,"monotonic":2345978.394973302,"hcas":{"mlx5_0":{"port_xmit_data":82837787607724,"port_rcv_data":84329129929405,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82795691663407,"port_rcv_data":82223109285950,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:01.026","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:01.040","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:01.053","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.58,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:01.066","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:01.078","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:01.089","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:01.099","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:01.109","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259141.9467247,"monotonic":2345979.395792526,"hcas":{"mlx5_0":{"port_xmit_data":82837798273162,"port_rcv_data":84330392903331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82796952286110,"port_rcv_data":82223119943423,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:02.046","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:02.060","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:02.074","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.26,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:02.086","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:02.097","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:02.108","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:02.118","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:02.128","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":147.39,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259142.9475224,"monotonic":2345980.396590308,"hcas":{"mlx5_0":{"port_xmit_data":82837808797370,"port_rcv_data":84331635062452,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82798196279537,"port_rcv_data":82223130493598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:03.046","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:03.060","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:03.073","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.82,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:03.086","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:03.097","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:03.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:03.119","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.84,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:03.129","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.38,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259143.9483094,"monotonic":2345981.397377374,"hcas":{"mlx5_0":{"port_xmit_data":82837819440376,"port_rcv_data":84332890640333,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913257657,"port_rcv_data":3200170516526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82799451661597,"port_rcv_data":82223141151980,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429686792,"port_rcv_data":2272901502041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:04.046","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:04.060","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:04.073","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.73,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:04.085","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:04.096","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:04.107","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:04.117","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:04.126","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.07,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259144.9491084,"monotonic":2345982.398176498,"hcas":{"mlx5_0":{"port_xmit_data":82837830217640,"port_rcv_data":84334068320313,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913267050,"port_rcv_data":3200180089646,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82800620886526,"port_rcv_data":82223161524173,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429696216,"port_rcv_data":2272911075161,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:05.029","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:05.043","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:05.056","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.45,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:05.068","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:05.079","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:05.090","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:05.100","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:05.110","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.39,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259145.9499135,"monotonic":2345983.398981771,"hcas":{"mlx5_0":{"port_xmit_data":82837841136080,"port_rcv_data":84335262005972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913290440,"port_rcv_data":3200204022446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82801789883690,"port_rcv_data":82223196356288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429719652,"port_rcv_data":2272935007961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:06.067","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:06.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:06.094","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.17,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:06.106","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:06.117","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:06.128","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:06.138","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:06.148","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259146.9506998,"monotonic":2345984.399767817,"hcas":{"mlx5_0":{"port_xmit_data":82837852132068,"port_rcv_data":84336500816358,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913346317,"port_rcv_data":3200261461166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82802971510095,"port_rcv_data":82223264745961,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429775654,"port_rcv_data":2272992446681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:07.032","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.84,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:07.046","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:07.060","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.83,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:07.072","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.94,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:07.083","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:07.094","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:07.104","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.93,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:07.113","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.36,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259147.9515176,"monotonic":2345985.400585604,"hcas":{"mlx5_0":{"port_xmit_data":82837863042934,"port_rcv_data":84337704807881,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913374357,"port_rcv_data":3200290180526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82804147082673,"port_rcv_data":82223304357362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429803616,"port_rcv_data":2273021166041,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:08.032","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:08.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:08.060","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.0,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:08.072","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:08.084","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:08.095","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:08.105","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:08.115","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.66,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259148.9523299,"monotonic":2345986.40139796,"hcas":{"mlx5_0":{"port_xmit_data":82837873944551,"port_rcv_data":84338907834369,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913402365,"port_rcv_data":3200318899886,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82805319035578,"port_rcv_data":82223343942219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429831578,"port_rcv_data":2273049885401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:09.051","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:09.065","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:09.078","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.97,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:09.091","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:09.102","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.68,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:09.113","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:09.123","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:09.133","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.39,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259149.9531267,"monotonic":2345987.402194506,"hcas":{"mlx5_0":{"port_xmit_data":82837884851165,"port_rcv_data":84340111460231,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913430761,"port_rcv_data":3200347619246,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82806496166685,"port_rcv_data":82223383579235,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555429859571,"port_rcv_data":2273078604761,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:10.034","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:10.047","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:10.061","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.42,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:10.073","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.39,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:10.085","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:10.095","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:10.105","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:10.115","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259150.9539368,"monotonic":2345988.403005064,"hcas":{"mlx5_0":{"port_xmit_data":82837895765206,"port_rcv_data":84341302702473,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913444773,"port_rcv_data":3200361978926,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82807672829969,"port_rcv_data":82223426527997,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555430240003,"port_rcv_data":2273110647641,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:11.054","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:11.068","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:11.081","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.43,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:11.093","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:11.104","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:11.115","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:11.126","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:11.136","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.69,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259151.9547439,"monotonic":2345989.403811517,"hcas":{"mlx5_0":{"port_xmit_data":82837907069157,"port_rcv_data":84342501895215,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438913799119,"port_rcv_data":3200379662126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82808855254078,"port_rcv_data":82223446327052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555430408720,"port_rcv_data":2273119489241,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:12.055","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:12.069","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:12.082","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.52,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:12.094","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.42,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:12.105","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:12.116","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:12.126","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:12.136","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.51,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259152.9555497,"monotonic":2345990.404617517,"hcas":{"mlx5_0":{"port_xmit_data":82837918766735,"port_rcv_data":84343694691559,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438914181271,"port_rcv_data":3200452875775,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82809974587826,"port_rcv_data":82223522138593,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555430633641,"port_rcv_data":2273183753079,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:13.072","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:13.085","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:13.099","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.48,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:13.111","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.18,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:13.122","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:13.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:13.143","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.72,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:13.153","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.93,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259153.9563384,"monotonic":2345991.405406235,"hcas":{"mlx5_0":{"port_xmit_data":82837930595670,"port_rcv_data":84344802107843,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438914437595,"port_rcv_data":3200482951855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82811050588780,"port_rcv_data":82223565759127,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555430965666,"port_rcv_data":2273215795959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:14.056","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:14.070","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:14.083","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.89,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:14.095","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:14.106","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:14.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:14.127","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.62,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:14.137","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.49,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259154.9571426,"monotonic":2345992.406210526,"hcas":{"mlx5_0":{"port_xmit_data":82837942487721,"port_rcv_data":84345811591368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438914554744,"port_rcv_data":3200502208855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82812043287725,"port_rcv_data":82223609610257,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555431306139,"port_rcv_data":2273247838839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:15.072","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:15.086","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.18,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:15.098","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.8,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:15.110","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.06,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:15.121","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:15.132","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:15.142","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.89,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:15.152","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.14,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259155.9579341,"monotonic":2345993.407002128,"hcas":{"mlx5_0":{"port_xmit_data":82837955493600,"port_rcv_data":84346767785857,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438914985597,"port_rcv_data":3200542672913,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82812956975714,"port_rcv_data":82223660232123,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555431525015,"port_rcv_data":2273285886253,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:16.060","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:16.074","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:16.088","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.23,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:16.100","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:16.112","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:16.123","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:16.133","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:16.143","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259156.958734,"monotonic":2345994.407801758,"hcas":{"mlx5_0":{"port_xmit_data":82837968051934,"port_rcv_data":84347685662316,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438915138629,"port_rcv_data":3200575564131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82813843965575,"port_rcv_data":82223693122185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555431607258,"port_rcv_data":2273306440263,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:17.060","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:17.074","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:17.087","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.63,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:17.100","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:17.112","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.47,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:17.123","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:17.133","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:17.143","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.21,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259157.9589524,"monotonic":2345995.408020188,"hcas":{"mlx5_0":{"port_xmit_data":82837980656584,"port_rcv_data":84348600048554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438915153183,"port_rcv_data":3200589923811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82814744162882,"port_rcv_data":82223720081896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555431621270,"port_rcv_data":2273320799943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:18.039","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:18.053","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:18.066","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.71,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:18.078","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:18.090","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.69,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:18.101","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:18.111","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:18.122","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.59,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259158.959772,"monotonic":2345996.408840123,"hcas":{"mlx5_0":{"port_xmit_data":82837993081944,"port_rcv_data":84349488221629,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438915153183,"port_rcv_data":3200589923811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82815632534419,"port_rcv_data":82223741370853,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555431808417,"port_rcv_data":2273329641543,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:19.059","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:19.073","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:19.086","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.85,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:19.098","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:19.109","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:19.119","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:19.129","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.63,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:19.139","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.55,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259159.9605758,"monotonic":2345997.409643897,"hcas":{"mlx5_0":{"port_xmit_data":82838005721968,"port_rcv_data":84350441999224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438915379173,"port_rcv_data":3200653687299,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82816522736036,"port_rcv_data":82223829637303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555432245796,"port_rcv_data":2273405513899,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:20.077","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:20.090","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:20.104","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.4,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:20.116","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.03,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:20.128","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:20.139","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:20.149","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:20.159","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.54,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259160.9613585,"monotonic":2345998.410426696,"hcas":{"mlx5_0":{"port_xmit_data":82838019118301,"port_rcv_data":84351397262984,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438915916558,"port_rcv_data":3200743631025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82817386429205,"port_rcv_data":82223911540616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555432432261,"port_rcv_data":2273474503513,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:21.059","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:21.073","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:21.086","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.7,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:21.098","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.3,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:21.109","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:21.120","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:21.131","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:21.141","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.66,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259161.9616551,"monotonic":2345999.410722644,"hcas":{"mlx5_0":{"port_xmit_data":82838032507159,"port_rcv_data":84352190124156,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438915960563,"port_rcv_data":3200787293611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82818135643528,"port_rcv_data":82223968583524,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555432476405,"port_rcv_data":2273518167399,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:22.063","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:22.077","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.09,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:22.090","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.46,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:22.102","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.96,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:22.114","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:22.125","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:22.135","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:22.145","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.13,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259162.9624488,"monotonic":2346000.411516747,"hcas":{"mlx5_0":{"port_xmit_data":82838045755432,"port_rcv_data":84352969760147,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438915998956,"port_rcv_data":3200825586091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82818878014980,"port_rcv_data":82224020108856,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555432513806,"port_rcv_data":2273556459879,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:23.062","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:23.077","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:23.090","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.51,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:23.102","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.45,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:23.114","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.64,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:23.125","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:23.135","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:23.145","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.55,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259163.963245,"monotonic":2346001.412312839,"hcas":{"mlx5_0":{"port_xmit_data":82838058384127,"port_rcv_data":84353614162344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438916041256,"port_rcv_data":3200868665562,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82819478683345,"port_rcv_data":82224096140399,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555432880009,"port_rcv_data":2273619969806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:24.063","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:24.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:24.089","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.46,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:24.102","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:24.113","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:24.124","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:24.134","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.2,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:24.144","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259164.964042,"monotonic":2346002.413110106,"hcas":{"mlx5_0":{"port_xmit_data":82838071201340,"port_rcv_data":84354232617484,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438916405878,"port_rcv_data":3200938968144,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82820027094895,"port_rcv_data":82224176240951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555433221699,"port_rcv_data":2273687523845,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:25.062","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:25.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:25.089","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.64,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:25.102","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:25.114","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.84,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:25.125","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:25.135","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.92,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:25.145","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.35,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259165.9648335,"monotonic":2346003.413901585,"hcas":{"mlx5_0":{"port_xmit_data":82838083943240,"port_rcv_data":84354850140935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438916813543,"port_rcv_data":3201014040239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82820569850933,"port_rcv_data":82224246067420,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555433277747,"port_rcv_data":2273744912855,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:26.063","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":133.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:26.077","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:26.090","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":130.77,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:26.103","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:26.114","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:26.124","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:26.134","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":131.19,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:26.144","gpu_util_pct":98.0,"memory_used_mib":80005.0,"power_w":128.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259166.9656336,"monotonic":2346004.414701499,"hcas":{"mlx5_0":{"port_xmit_data":82838096561640,"port_rcv_data":84355484456340,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438917030430,"port_rcv_data":3201066010727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82821152746636,"port_rcv_data":82224319170914,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555433678748,"port_rcv_data":2273805724943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:27.062","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:27.076","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:27.089","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.43,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:27.101","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:27.112","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:27.122","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:27.132","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.7,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:27.142","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.15,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259167.966434,"monotonic":2346005.415502083,"hcas":{"mlx5_0":{"port_xmit_data":82838109050928,"port_rcv_data":84356068503058,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438917233263,"port_rcv_data":3201108410523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82821696023276,"port_rcv_data":82224382696801,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555434071487,"port_rcv_data":2273856966339,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:28.067","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":137.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:28.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:28.094","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":126.72,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:28.107","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.31,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:28.118","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:28.130","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:28.140","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:28.150","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":125.57,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259168.9672256,"monotonic":2346006.416293657,"hcas":{"mlx5_0":{"port_xmit_data":82838121659817,"port_rcv_data":84356678568291,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438917634713,"port_rcv_data":3201181823681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82822232829791,"port_rcv_data":82224464115900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555434438883,"port_rcv_data":2273926746659,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:29.048","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:29.062","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:29.075","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.23,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:29.087","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":127.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:29.098","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:29.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:29.119","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.45,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:29.129","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":126.75,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259169.968037,"monotonic":2346007.417104931,"hcas":{"mlx5_0":{"port_xmit_data":82838134034293,"port_rcv_data":84357235354365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438917802098,"port_rcv_data":3201200913295,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82822769840669,"port_rcv_data":82224508410936,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555434822648,"port_rcv_data":2273958310583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:30.067","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.83,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:30.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:30.094","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.4,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:30.108","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:30.119","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:30.130","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:30.141","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.53,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:30.151","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":130.18,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259170.9688296,"monotonic":2346008.417897717,"hcas":{"mlx5_0":{"port_xmit_data":82838146860467,"port_rcv_data":84357875225747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438918420160,"port_rcv_data":3201302546493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82823308373836,"port_rcv_data":82224595849847,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555434900628,"port_rcv_data":2274033418361,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:31.066","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:31.080","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:31.093","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.02,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:31.106","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.08,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:31.118","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:31.128","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:31.138","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.78,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:31.149","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.05,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259171.96963,"monotonic":2346009.41869801,"hcas":{"mlx5_0":{"port_xmit_data":82838159408101,"port_rcv_data":84358473059258,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438918800561,"port_rcv_data":3201365879723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82823842365325,"port_rcv_data":82224671415169,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555435336411,"port_rcv_data":2274096751591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:32.050","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":134.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:32.064","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.9,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:32.077","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.65,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:32.089","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:32.100","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:32.111","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:32.121","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.88,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:32.131","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.95,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259172.9704473,"monotonic":2346010.419515308,"hcas":{"mlx5_0":{"port_xmit_data":82838171866602,"port_rcv_data":84359080410322,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438919022909,"port_rcv_data":3201434364780,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82824381652880,"port_rcv_data":82224760901756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555435712239,"port_rcv_data":2274174076948,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:33.070","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:33.084","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:33.097","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.43,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:33.110","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":126.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:33.121","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":130.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:33.132","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:33.142","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.15,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:33.152","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.01,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259173.971249,"monotonic":2346011.420316927,"hcas":{"mlx5_0":{"port_xmit_data":82838184323771,"port_rcv_data":84359692613939,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438919245659,"port_rcv_data":3201507569523,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82824920946091,"port_rcv_data":82224855212735,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555436110837,"port_rcv_data":2274256157695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:34.087","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:34.101","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:34.114","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":125.65,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:34.127","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:34.138","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":128.48,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:34.149","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:34.160","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.79,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:34.170","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":127.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259174.972031,"monotonic":2346012.421099128,"hcas":{"mlx5_0":{"port_xmit_data":82838197026322,"port_rcv_data":84360310393273,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438919797227,"port_rcv_data":3201591894008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82825455470920,"port_rcv_data":82224933913757,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555436341803,"port_rcv_data":2274322765748,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:35.071","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:35.085","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":132.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:35.098","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":129.26,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:35.110","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:35.121","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:35.131","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:35.141","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.52,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:35.151","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.44,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259175.972842,"monotonic":2346013.421909811,"hcas":{"mlx5_0":{"port_xmit_data":82838209467840,"port_rcv_data":84360911117743,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438920001455,"port_rcv_data":3201653683606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82825995616669,"port_rcv_data":82225015541095,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555436700891,"port_rcv_data":2274392152897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:36.091","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":136.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:36.105","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.27,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:36.119","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.2,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:36.131","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:36.142","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":129.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:36.153","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:36.163","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:36.173","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":126.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259176.9736285,"monotonic":2346014.422696472,"hcas":{"mlx5_0":{"port_xmit_data":82838222149140,"port_rcv_data":84361525092018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438920347554,"port_rcv_data":3201721661413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82826540295282,"port_rcv_data":82225097082017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555437041953,"port_rcv_data":2274461373581,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:37.072","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":136.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:37.086","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":131.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:37.099","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.36,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:37.110","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":128.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:37.122","gpu_util_pct":98.0,"memory_used_mib":79875.0,"power_w":129.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:37.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:37.143","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":131.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:37.153","gpu_util_pct":98.0,"memory_used_mib":80005.0,"power_w":126.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259177.974428,"monotonic":2346015.423495718,"hcas":{"mlx5_0":{"port_xmit_data":82838235420759,"port_rcv_data":84362324019730,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438920560586,"port_rcv_data":3201787793597,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82827275318037,"port_rcv_data":82225194013839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555437559715,"port_rcv_data":2274545187665,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:38.072","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:38.086","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:38.099","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.79,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:38.111","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:38.122","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:38.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:38.143","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:38.153","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.83,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259178.9752314,"monotonic":2346016.424299306,"hcas":{"mlx5_0":{"port_xmit_data":82838249085170,"port_rcv_data":84363132143444,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438921074892,"port_rcv_data":3201857652897,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82828012069130,"port_rcv_data":82225259415103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555437777165,"port_rcv_data":2274597429514,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:39.100","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:39.114","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:39.127","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.42,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:39.140","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:39.151","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:39.162","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.91,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:39.172","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:39.182","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259179.9760032,"monotonic":2346017.425071006,"hcas":{"mlx5_0":{"port_xmit_data":82838262684357,"port_rcv_data":84363923468497,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438921583075,"port_rcv_data":3201913484249,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82828747798097,"port_rcv_data":82225310516806,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555437988492,"port_rcv_data":2274635447632,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:40.096","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:40.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:40.123","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.3,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:40.136","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":132.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:40.148","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.17,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:40.159","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:40.170","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.08,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:40.180","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259180.9767802,"monotonic":2346018.425848171,"hcas":{"mlx5_0":{"port_xmit_data":82838275928916,"port_rcv_data":84364727580905,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438921793007,"port_rcv_data":3201977124571,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82829489798516,"port_rcv_data":82225404462911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555438536541,"port_rcv_data":2274716212794,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:41.057","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":140.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:41.072","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:41.084","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.95,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:41.096","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:41.108","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:41.118","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:41.129","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.36,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:41.139","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.28,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259181.9776025,"monotonic":2346019.426670605,"hcas":{"mlx5_0":{"port_xmit_data":82838288795412,"port_rcv_data":84365680212620,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438922305638,"port_rcv_data":3202046085039,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82830375148455,"port_rcv_data":82225469356154,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555438722789,"port_rcv_data":2274768733232,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:42.075","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:42.089","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:42.102","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.43,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:42.114","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.38,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:42.125","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:42.136","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:42.146","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:42.156","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259182.978412,"monotonic":2346020.427479941,"hcas":{"mlx5_0":{"port_xmit_data":82838301322283,"port_rcv_data":84366598956158,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438922497885,"port_rcv_data":3202084309042,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82831255637350,"port_rcv_data":82225537026866,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555439233219,"port_rcv_data":2274824018118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:43.078","gpu_util_pct":98.0,"memory_used_mib":79997.0,"power_w":141.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:43.092","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:43.105","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":137.23,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:43.117","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:43.129","gpu_util_pct":98.0,"memory_used_mib":79875.0,"power_w":136.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:43.140","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:43.150","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.27,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:43.160","gpu_util_pct":98.0,"memory_used_mib":80005.0,"power_w":135.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259183.9792247,"monotonic":2346021.428292223,"hcas":{"mlx5_0":{"port_xmit_data":82838313302124,"port_rcv_data":84367734082238,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438922824501,"port_rcv_data":3202241032424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82832234264156,"port_rcv_data":82225723186711,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555439880716,"port_rcv_data":2274998456899,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:44.095","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:44.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:44.122","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.79,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:44.135","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:44.146","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:44.157","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:44.167","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:44.177","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.13,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259184.9800181,"monotonic":2346022.42908613,"hcas":{"mlx5_0":{"port_xmit_data":82838325364933,"port_rcv_data":84368737907589,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438923295143,"port_rcv_data":3202267557224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82833211948132,"port_rcv_data":82225743659776,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555440040118,"port_rcv_data":2275007298499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:45.060","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:45.074","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:45.087","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.82,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:45.100","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.52,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:45.111","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.57,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:45.122","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:45.132","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:45.142","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259185.9808443,"monotonic":2346023.42991202,"hcas":{"mlx5_0":{"port_xmit_data":82838337344400,"port_rcv_data":84369740267347,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438923605530,"port_rcv_data":3202285240424,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82834195087976,"port_rcv_data":82225773018839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555440354908,"port_rcv_data":2275024981699,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:46.078","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:46.092","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:46.105","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.2,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:46.118","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:46.129","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:46.140","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:46.150","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.53,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:46.160","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.45,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259186.9816554,"monotonic":2346024.430723078,"hcas":{"mlx5_0":{"port_xmit_data":82838349177857,"port_rcv_data":84370729318365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438923919204,"port_rcv_data":3202302923839,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82835166783271,"port_rcv_data":82225802123040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555440660769,"port_rcv_data":2275042526616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:47.078","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:47.092","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:47.105","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.52,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:47.117","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:47.128","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:47.139","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:47.149","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:47.159","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.86,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259187.982462,"monotonic":2346025.431529937,"hcas":{"mlx5_0":{"port_xmit_data":82838360795680,"port_rcv_data":84371710151823,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438924069151,"port_rcv_data":3202311627949,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82836141107735,"port_rcv_data":82225839770916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555441105139,"port_rcv_data":2275068636636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:48.120","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:48.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:48.147","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.6,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:48.160","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:48.171","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:48.182","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:48.193","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.94,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:48.203","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.0,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259188.9832263,"monotonic":2346026.432294231,"hcas":{"mlx5_0":{"port_xmit_data":82838372888170,"port_rcv_data":84372714040092,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438924511397,"port_rcv_data":3202337738555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82837118941417,"port_rcv_data":82225860499952,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555441260402,"port_rcv_data":2275077685530,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:49.100","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:49.114","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:49.128","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.99,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:49.140","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:49.152","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:49.163","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:49.173","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:49.183","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.04,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259189.984012,"monotonic":2346027.433079691,"hcas":{"mlx5_0":{"port_xmit_data":82838384300333,"port_rcv_data":84373791061266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438924669698,"port_rcv_data":3202346788035,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82838187971656,"port_rcv_data":82225898921273,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555441727743,"port_rcv_data":2275104831660,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:50.086","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:50.100","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:50.114","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.5,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:50.126","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.77,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:50.137","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:50.148","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:50.159","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:50.169","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.3,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259190.9848096,"monotonic":2346028.433877438,"hcas":{"mlx5_0":{"port_xmit_data":82838395981881,"port_rcv_data":84374912867005,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925134063,"port_rcv_data":3202373934751,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82839282838173,"port_rcv_data":82225919205490,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555441880139,"port_rcv_data":2275113880554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:51.101","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:51.115","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:51.129","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":140.4,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:51.141","gpu_util_pct":98.0,"memory_used_mib":79963.0,"power_w":139.75,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:51.153","gpu_util_pct":98.0,"memory_used_mib":79875.0,"power_w":142.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:51.163","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:51.174","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":144.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:51.184","gpu_util_pct":99.0,"memory_used_mib":80005.0,"power_w":140.52,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259191.9855926,"monotonic":2346029.434660585,"hcas":{"mlx5_0":{"port_xmit_data":82838407020336,"port_rcv_data":84376078065454,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925443985,"port_rcv_data":3202392032849,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82840429653220,"port_rcv_data":82225948037196,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442188418,"port_rcv_data":2275131978066,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:52.069","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:52.083","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:52.096","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.26,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:52.109","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:52.120","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.94,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:52.131","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:52.141","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.73,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:52.151","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.94,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259192.9864092,"monotonic":2346030.435477205,"hcas":{"mlx5_0":{"port_xmit_data":82838418129265,"port_rcv_data":84377262220492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82841596196622,"port_rcv_data":82225967913206,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:53.066","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:53.081","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:53.093","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.51,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:53.105","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:53.117","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:53.127","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:53.137","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:53.147","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.76,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259193.9872303,"monotonic":2346031.436298157,"hcas":{"mlx5_0":{"port_xmit_data":82838428482374,"port_rcv_data":84378470850106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82842805229665,"port_rcv_data":82225978297573,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:54.086","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:54.100","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:54.113","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.44,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:54.125","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:54.136","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:54.147","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:54.157","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.82,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:54.167","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259194.9880228,"monotonic":2346032.437090639,"hcas":{"mlx5_0":{"port_xmit_data":82838438784036,"port_rcv_data":84379696469418,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82844031095060,"port_rcv_data":82225988616805,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:55.104","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:55.118","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:55.131","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.28,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:55.143","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:55.155","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:55.165","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:55.176","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:55.186","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259195.9888008,"monotonic":2346033.437868837,"hcas":{"mlx5_0":{"port_xmit_data":82838449174815,"port_rcv_data":84380920777527,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82845255095106,"port_rcv_data":82225999017408,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:56.104","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:56.118","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.42,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:56.131","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.68,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:56.144","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:56.155","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:56.166","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:56.177","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.44,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:56.186","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259196.9895706,"monotonic":2346034.438638723,"hcas":{"mlx5_0":{"port_xmit_data":82838459542840,"port_rcv_data":84382149518943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82846483986028,"port_rcv_data":82226009406804,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:57.070","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:57.083","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.51,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:57.097","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.14,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:57.109","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:57.121","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:57.132","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:57.142","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.1,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:57.152","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.11,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259197.990387,"monotonic":2346035.439454927,"hcas":{"mlx5_0":{"port_xmit_data":82838469901641,"port_rcv_data":84383376667009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82847710878720,"port_rcv_data":82226019770959,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:58.090","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:58.104","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:58.117","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.09,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:58.129","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:58.140","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:58.151","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:58.161","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.41,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:58.171","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259198.991188,"monotonic":2346036.440255904,"hcas":{"mlx5_0":{"port_xmit_data":82838480212812,"port_rcv_data":84384596414989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82848930219252,"port_rcv_data":82226030102520,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:39:59.091","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:39:59.105","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:39:59.118","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.98,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:39:59.131","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:39:59.142","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:39:59.154","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.73,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:39:59.164","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:39:59.174","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259199.991982,"monotonic":2346037.441050113,"hcas":{"mlx5_0":{"port_xmit_data":82838490570839,"port_rcv_data":84385823649004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82850155766583,"port_rcv_data":82226040452606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:00.072","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:00.085","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.84,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:00.098","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.68,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:00.110","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:00.122","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":143.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:00.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:00.143","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:00.152","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":145.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259200.99281,"monotonic":2346038.441877606,"hcas":{"mlx5_0":{"port_xmit_data":82838500923475,"port_rcv_data":84387050017464,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82851384687059,"port_rcv_data":82226050846710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:01.075","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:01.089","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:01.102","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.95,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:01.114","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.57,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:01.125","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:01.136","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:01.146","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:01.156","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.14,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259201.9936218,"monotonic":2346039.442689632,"hcas":{"mlx5_0":{"port_xmit_data":82838511239562,"port_rcv_data":84388268545782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82852603234326,"port_rcv_data":82226061173572,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:02.113","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:02.129","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:02.142","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.8,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:02.154","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:02.165","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:02.175","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:02.186","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:02.195","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.26,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259202.9944112,"monotonic":2346040.443478829,"hcas":{"mlx5_0":{"port_xmit_data":82838521865001,"port_rcv_data":84389464046396,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925752947,"port_rcv_data":3202410130732,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82853798320012,"port_rcv_data":82226071816280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442340752,"port_rcv_data":2275141026891,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:03.095","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:03.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:03.122","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.41,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:03.135","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:03.146","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:03.154","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:03.163","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:03.171","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259203.9949553,"monotonic":2346041.444023041,"hcas":{"mlx5_0":{"port_xmit_data":82838533012745,"port_rcv_data":84390586100870,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925776553,"port_rcv_data":3202434063532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82854895997506,"port_rcv_data":82226106891269,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442364204,"port_rcv_data":2275164959691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:04.095","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:04.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:04.122","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.51,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:04.135","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.71,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:04.146","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.72,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:04.157","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:04.167","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:04.177","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.18,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259204.995762,"monotonic":2346042.444829964,"hcas":{"mlx5_0":{"port_xmit_data":82838544288179,"port_rcv_data":84391700078048,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925790643,"port_rcv_data":3202448423212,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82855994256344,"port_rcv_data":82226132518002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442378448,"port_rcv_data":2275179319371,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:05.095","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:05.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.43,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:05.123","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.51,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:05.135","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:05.146","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:05.157","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.89,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:05.167","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:05.177","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.61,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259205.9965675,"monotonic":2346043.445635381,"hcas":{"mlx5_0":{"port_xmit_data":82838555702869,"port_rcv_data":84392828392008,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925804655,"port_rcv_data":3202462782892,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82857110546365,"port_rcv_data":82226158322118,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442392445,"port_rcv_data":2275193679051,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:06.095","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:06.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:06.123","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.83,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:06.135","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.27,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:06.146","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:06.158","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:06.168","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:06.178","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.72,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259206.9973755,"monotonic":2346044.44644302,"hcas":{"mlx5_0":{"port_xmit_data":82838567039010,"port_rcv_data":84393953601549,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925825533,"port_rcv_data":3202484322412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82858212639079,"port_rcv_data":82226191165222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442413664,"port_rcv_data":2275215717175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:07.095","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:07.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:07.122","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.16,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:07.134","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:07.145","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.96,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:07.156","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:07.166","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:07.176","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.22,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259207.9981835,"monotonic":2346045.447251416,"hcas":{"mlx5_0":{"port_xmit_data":82838578288834,"port_rcv_data":84395072340345,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925846706,"port_rcv_data":3202505861932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82859308887837,"port_rcv_data":82226223954663,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442434481,"port_rcv_data":2275236758091,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:08.115","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:08.129","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:08.142","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.38,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:08.154","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:08.166","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:08.177","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:08.187","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.16,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:08.197","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.53,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259208.9989734,"monotonic":2346046.448041207,"hcas":{"mlx5_0":{"port_xmit_data":82838589519401,"port_rcv_data":84396168555815,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438925846706,"port_rcv_data":3202505861932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82860405268477,"port_rcv_data":82226243354085,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442591434,"port_rcv_data":2275244954211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:09.079","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:09.093","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.96,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:09.106","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.62,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:09.119","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:09.130","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:09.142","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:09.152","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:09.162","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.39,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259209.9998014,"monotonic":2346047.448869044,"hcas":{"mlx5_0":{"port_xmit_data":82838601475751,"port_rcv_data":84397251114052,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438926227154,"port_rcv_data":3202563781386,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82861431069393,"port_rcv_data":82226310984713,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555442923645,"port_rcv_data":2275300918125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:10.116","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:10.129","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:10.143","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.45,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:10.156","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:10.167","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:10.178","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:10.188","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.11,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:10.198","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.75,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259211.0005865,"monotonic":2346048.44965441,"hcas":{"mlx5_0":{"port_xmit_data":82838614273953,"port_rcv_data":84398183315027,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438926573811,"port_rcv_data":3202606134644,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82862320341069,"port_rcv_data":82226359634792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555443209868,"port_rcv_data":2275336972440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:11.099","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:11.113","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.8,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:11.127","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.21,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:11.139","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.97,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:11.151","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.4,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:11.162","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:11.173","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:11.183","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.62,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259212.0013835,"monotonic":2346049.45045134,"hcas":{"mlx5_0":{"port_xmit_data":82838626629011,"port_rcv_data":84399120495250,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438926634835,"port_rcv_data":3202664990138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82863199385059,"port_rcv_data":82226430817331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555443270504,"port_rcv_data":2275395827934,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:12.120","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:12.134","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:12.148","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.87,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:12.161","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.15,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:12.172","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.73,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:12.183","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:12.193","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.7,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:12.203","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.45,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259213.0021613,"monotonic":2346050.451229188,"hcas":{"mlx5_0":{"port_xmit_data":82838639085679,"port_rcv_data":84400038930782,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438926664843,"port_rcv_data":3202694389014,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82864087742391,"port_rcv_data":82226472654298,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555443301024,"port_rcv_data":2275425284553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:13.082","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:13.097","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:13.111","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.52,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:13.123","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:13.135","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:13.146","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:13.156","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:13.166","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.22,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259214.0029764,"monotonic":2346051.4520444,"hcas":{"mlx5_0":{"port_xmit_data":82838651423161,"port_rcv_data":84400934078953,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438926679196,"port_rcv_data":3202708748909,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82864971088250,"port_rcv_data":82226515649655,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555443575575,"port_rcv_data":2275455985365,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:14.084","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:14.098","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:14.111","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.23,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:14.124","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:14.135","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:14.146","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:14.157","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:14.167","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.76,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259215.0037897,"monotonic":2346052.452857446,"hcas":{"mlx5_0":{"port_xmit_data":82838664219124,"port_rcv_data":84401860031719,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438927073717,"port_rcv_data":3202747502217,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82865857108026,"port_rcv_data":82226552861822,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555443788561,"port_rcv_data":2275480760592,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:15.085","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:15.099","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.78,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:15.112","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.81,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:15.124","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:15.136","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:15.146","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:15.157","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.76,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:15.166","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259216.0046148,"monotonic":2346053.453682847,"hcas":{"mlx5_0":{"port_xmit_data":82838676729636,"port_rcv_data":84402781738329,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438927358762,"port_rcv_data":3202782718010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82866744969273,"port_rcv_data":82226597957951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555443980528,"port_rcv_data":2275513612377,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:16.103","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:16.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:16.130","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.32,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:16.142","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:16.153","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:16.164","gpu_util_pct":98.0,"memory_used_mib":79959.0,"power_w":143.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:16.174","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:16.184","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.92,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259217.005423,"monotonic":2346054.454490984,"hcas":{"mlx5_0":{"port_xmit_data":82838688324401,"port_rcv_data":84403758783653,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438927358762,"port_rcv_data":3202782718010,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82867720592961,"port_rcv_data":82226636077922,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555444503436,"port_rcv_data":2275540137177,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:17.103","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:17.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:17.130","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.98,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:17.142","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:17.154","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.76,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:17.164","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:17.175","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:17.184","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.37,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259218.0062299,"monotonic":2346055.455297456,"hcas":{"mlx5_0":{"port_xmit_data":82838700429886,"port_rcv_data":84404822584854,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438927895403,"port_rcv_data":3202872116952,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82868697013105,"port_rcv_data":82226719397332,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555444731829,"port_rcv_data":2275611852919,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:18.107","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.65,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:18.121","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.39,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:18.134","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.52,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:18.146","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:18.158","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.74,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:18.169","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:18.179","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:18.190","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.03,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259219.0070343,"monotonic":2346056.456102273,"hcas":{"mlx5_0":{"port_xmit_data":82838712227745,"port_rcv_data":84405851809602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438928092904,"port_rcv_data":3202924152138,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82869674417864,"port_rcv_data":82226774301538,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555444775384,"port_rcv_data":2275655046505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:19.127","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:19.141","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:19.155","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.5,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:19.168","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:19.179","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:19.190","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:19.201","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:19.210","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.27,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259220.0078177,"monotonic":2346057.456885453,"hcas":{"mlx5_0":{"port_xmit_data":82838723782335,"port_rcv_data":84406867719239,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438928137048,"port_rcv_data":3202967482638,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82870649161428,"port_rcv_data":82226829182486,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555444820551,"port_rcv_data":2275698375833,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:20.090","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:20.104","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.19,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:20.117","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.78,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:20.130","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:20.141","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:20.152","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:20.163","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.8,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:20.173","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.56,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259221.0086398,"monotonic":2346058.457707724,"hcas":{"mlx5_0":{"port_xmit_data":82838735854202,"port_rcv_data":84407857026088,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438928180913,"port_rcv_data":3203010830724,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82871593255780,"port_rcv_data":82226884592018,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555444864199,"port_rcv_data":2275741722619,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:21.107","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:21.120","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:21.134","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.14,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:21.146","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:21.158","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:21.169","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:21.179","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.54,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:21.190","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.35,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259222.009444,"monotonic":2346059.458512113,"hcas":{"mlx5_0":{"port_xmit_data":82838748360810,"port_rcv_data":84408791815548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438928225708,"port_rcv_data":3203054161224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82872484605609,"port_rcv_data":82226940428412,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555444908901,"port_rcv_data":2275785051947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:22.090","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:22.104","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:22.118","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.93,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:22.130","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:22.142","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:22.153","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:22.163","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:22.173","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.73,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259223.010267,"monotonic":2346060.459335086,"hcas":{"mlx5_0":{"port_xmit_data":82838761055797,"port_rcv_data":84409721759389,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438928437593,"port_rcv_data":3203091989930,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82873375094642,"port_rcv_data":82226999546771,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555445293673,"port_rcv_data":2275831723553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:23.108","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":139.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:23.122","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":137.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:23.135","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.11,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:23.147","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.13,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:23.159","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:23.170","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:23.180","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:23.190","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":133.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259224.0110738,"monotonic":2346061.460141667,"hcas":{"mlx5_0":{"port_xmit_data":82838774080523,"port_rcv_data":84410664180801,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438929004366,"port_rcv_data":3203143181548,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82874268319499,"port_rcv_data":82227045670752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555445487562,"port_rcv_data":2275865231351,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:24.127","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:24.141","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:24.154","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.07,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:24.166","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.48,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:24.178","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:24.189","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:24.199","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.87,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:24.209","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.95,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259225.0118628,"monotonic":2346062.460930641,"hcas":{"mlx5_0":{"port_xmit_data":82838786650572,"port_rcv_data":84411590039477,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438929036854,"port_rcv_data":3203173111310,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82875163040824,"port_rcv_data":82227122863603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555446176289,"port_rcv_data":2275929904772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:25.092","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:25.106","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:25.120","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":132.47,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:25.132","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:25.143","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":135.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:25.154","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:25.164","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:25.174","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.16,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259226.0126872,"monotonic":2346063.461754965,"hcas":{"mlx5_0":{"port_xmit_data":82838799721839,"port_rcv_data":84412666528467,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438929654746,"port_rcv_data":3203360559710,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82876052350700,"port_rcv_data":82227305567391,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555446500831,"port_rcv_data":2276100084126,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:26.127","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:26.141","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:26.155","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.3,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:26.168","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":134.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:26.179","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":133.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:26.190","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.34,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:26.201","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:26.211","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":132.21,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259227.0134726,"monotonic":2346064.462540462,"hcas":{"mlx5_0":{"port_xmit_data":82838812050640,"port_rcv_data":84413638529211,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438929979611,"port_rcv_data":3203401968492,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82876983285834,"port_rcv_data":82227359208998,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555446844792,"port_rcv_data":2276141700195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:27.127","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":142.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:27.140","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:27.154","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.37,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:27.166","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.4,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:27.177","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:27.189","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:27.199","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:27.209","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.33,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259228.0142643,"monotonic":2346065.463332131,"hcas":{"mlx5_0":{"port_xmit_data":82838824206778,"port_rcv_data":84414664091029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438930456546,"port_rcv_data":3203442852972,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82877968150061,"port_rcv_data":82227394110555,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555447015896,"port_rcv_data":2276164901475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:28.131","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:28.144","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:28.158","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.32,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:28.170","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.91,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:28.182","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.06,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:28.193","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.32,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:28.203","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.38,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:28.213","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.5,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259229.015048,"monotonic":2346066.464115738,"hcas":{"mlx5_0":{"port_xmit_data":82838836016866,"port_rcv_data":84415668877390,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438930622086,"port_rcv_data":3203466132076,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82878951373619,"port_rcv_data":82227446804426,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555447493963,"port_rcv_data":2276205940518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:29.095","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:29.109","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:29.122","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.15,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:29.134","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.65,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:29.146","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:29.157","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:29.167","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.19,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:29.177","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259230.0158708,"monotonic":2346067.464938716,"hcas":{"mlx5_0":{"port_xmit_data":82838849633361,"port_rcv_data":84416666453397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438931098447,"port_rcv_data":3203507248952,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82879908774967,"port_rcv_data":82227483238692,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555447661688,"port_rcv_data":2276229219252,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:30.115","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":144.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:30.129","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:30.142","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.93,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:30.154","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:30.166","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:30.177","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:30.187","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.29,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:30.197","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.93,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259231.0166771,"monotonic":2346068.46574497,"hcas":{"mlx5_0":{"port_xmit_data":82838862742942,"port_rcv_data":84417695147639,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438931297979,"port_rcv_data":3203544948718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82880898345945,"port_rcv_data":82227551723681,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555448153116,"port_rcv_data":2276284816754,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:31.097","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:31.111","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:31.125","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.38,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:31.137","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:31.149","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:31.160","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:31.171","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:31.181","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.19,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259232.0175004,"monotonic":2346069.466568194,"hcas":{"mlx5_0":{"port_xmit_data":82838875712523,"port_rcv_data":84418774715446,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438931773813,"port_rcv_data":3203571680979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82881951363252,"port_rcv_data":82227573212911,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555448324220,"port_rcv_data":2276293796505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:32.099","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:32.113","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:32.126","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.81,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:32.138","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:32.150","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:32.160","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:32.171","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:32.181","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259233.0183234,"monotonic":2346070.467391446,"hcas":{"mlx5_0":{"port_xmit_data":82838888109953,"port_rcv_data":84419853377830,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438931935029,"port_rcv_data":3203585800501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82883016340602,"port_rcv_data":82227623056040,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555448806224,"port_rcv_data":2276331554728,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:33.099","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:33.113","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:33.127","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.02,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:33.139","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:33.150","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.66,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:33.160","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:33.171","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.17,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:33.181","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.82,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259234.0191472,"monotonic":2346071.468215184,"hcas":{"mlx5_0":{"port_xmit_data":82838900984258,"port_rcv_data":84421066985136,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438932503708,"port_rcv_data":3203706292307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82884111523808,"port_rcv_data":82227732504583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555449054503,"port_rcv_data":2276428255179,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:34.100","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:34.114","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:34.128","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.04,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:34.140","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:34.152","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:34.163","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:34.173","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:34.183","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259235.0199623,"monotonic":2346072.469030389,"hcas":{"mlx5_0":{"port_xmit_data":82838913488747,"port_rcv_data":84422143984399,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438932527144,"port_rcv_data":3203730225107,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82885164670218,"port_rcv_data":82227786330746,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555449385273,"port_rcv_data":2276469611499,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:35.136","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:35.149","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:35.163","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.04,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:35.175","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:35.187","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:35.198","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.95,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:35.208","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:35.218","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.09,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259236.0207393,"monotonic":2346073.469807552,"hcas":{"mlx5_0":{"port_xmit_data":82838926329615,"port_rcv_data":84423226640671,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438932988595,"port_rcv_data":3203770952679,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82886206896248,"port_rcv_data":82227822049608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555449551975,"port_rcv_data":2276492967480,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:36.101","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:36.115","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.95,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:36.129","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.63,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:36.141","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.72,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:36.152","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:36.163","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:36.174","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:36.184","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":145.08,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259237.0215547,"monotonic":2346074.470622722,"hcas":{"mlx5_0":{"port_xmit_data":82838938912574,"port_rcv_data":84424294371951,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933157653,"port_rcv_data":3203794101943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82887252170636,"port_rcv_data":82227866375061,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555449863029,"port_rcv_data":2276524905243,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:37.119","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":148.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:37.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:37.147","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.42,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:37.159","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:37.170","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:37.181","gpu_util_pct":97.0,"memory_used_mib":79959.0,"power_w":149.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:37.191","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.51,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:37.201","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":146.2,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259238.0223513,"monotonic":2346075.471419475,"hcas":{"mlx5_0":{"port_xmit_data":82838951514234,"port_rcv_data":84425369561789,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933171805,"port_rcv_data":3203808461623,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82888311640075,"port_rcv_data":82227900502613,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555449885380,"port_rcv_data":2276547641403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:38.103","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:38.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.87,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:38.130","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.34,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:38.142","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:38.153","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.99,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:38.164","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:38.174","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:38.184","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259239.0229433,"monotonic":2346076.472011494,"hcas":{"mlx5_0":{"port_xmit_data":82838964036915,"port_rcv_data":84426449654607,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933199922,"port_rcv_data":3203837180983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82889362560054,"port_rcv_data":82227934547501,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555449905251,"port_rcv_data":2276567984283,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:39.103","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:39.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:39.130","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.98,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:39.142","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:39.154","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:39.164","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:39.175","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.91,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:39.184","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.47,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259240.023756,"monotonic":2346077.472824334,"hcas":{"mlx5_0":{"port_xmit_data":82838976555340,"port_rcv_data":84427512632606,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933209377,"port_rcv_data":3203846754103,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82890417644869,"port_rcv_data":82227963972125,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555450035792,"port_rcv_data":2276584908723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:40.103","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":149.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:40.117","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:40.131","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.58,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:40.144","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:40.155","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.54,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:40.166","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:40.176","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.34,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:40.187","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259241.0245702,"monotonic":2346078.473638421,"hcas":{"mlx5_0":{"port_xmit_data":82838989405783,"port_rcv_data":84428612820812,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933448030,"port_rcv_data":3203883241895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82891480171621,"port_rcv_data":82228020630047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555450474148,"port_rcv_data":2276628840115,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:41.105","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:41.119","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:41.132","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.22,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:41.144","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.57,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:41.156","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:41.166","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:41.177","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:41.186","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.51,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259242.0253825,"monotonic":2346079.474450598,"hcas":{"mlx5_0":{"port_xmit_data":82839002324616,"port_rcv_data":84429712990473,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933935040,"port_rcv_data":3203936344603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82892529345674,"port_rcv_data":82228071364542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555450671633,"port_rcv_data":2276667146603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:42.104","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:42.118","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:42.131","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.98,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:42.144","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.81,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:42.155","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:42.166","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:42.176","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:42.186","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259243.0261986,"monotonic":2346080.475266812,"hcas":{"mlx5_0":{"port_xmit_data":82839014830919,"port_rcv_data":84430794513249,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933964909,"port_rcv_data":3203965751319,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82893581810686,"port_rcv_data":82228113342625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555450701855,"port_rcv_data":2276696554491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:43.143","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:43.157","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.81,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:43.170","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.28,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:43.181","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:43.192","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:43.202","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:43.212","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:43.222","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":145.13,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259244.026991,"monotonic":2346081.476058385,"hcas":{"mlx5_0":{"port_xmit_data":82839027321063,"port_rcv_data":84431874829605,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438933995552,"port_rcv_data":3203995294949,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82894632483809,"port_rcv_data":82228155359380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555450732529,"port_rcv_data":2276726098121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:44.127","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:44.141","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:44.154","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.04,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:44.166","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:44.177","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:44.188","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:44.198","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:44.207","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.49,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259245.0277991,"monotonic":2346082.476866841,"hcas":{"mlx5_0":{"port_xmit_data":82839039795756,"port_rcv_data":84432961660326,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438934038751,"port_rcv_data":3204036219209,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82895677644945,"port_rcv_data":82228216109433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555450783137,"port_rcv_data":2276774408219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:45.127","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.05,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:45.140","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:45.154","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.45,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:45.167","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.23,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:45.179","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:45.190","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:45.200","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:45.211","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.46,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259246.028608,"monotonic":2346083.477675586,"hcas":{"mlx5_0":{"port_xmit_data":82839052338966,"port_rcv_data":84434057643143,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438934082817,"port_rcv_data":3204079124475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82896729508025,"port_rcv_data":82228264136895,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555450819546,"port_rcv_data":2276809927647,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:46.127","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:46.142","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.2,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:46.155","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.48,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:46.167","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.51,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:46.178","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:46.189","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:46.200","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:46.209","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259247.0294213,"monotonic":2346084.478489321,"hcas":{"mlx5_0":{"port_xmit_data":82839064839098,"port_rcv_data":84435111613401,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438934082817,"port_rcv_data":3204079124475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82897786858230,"port_rcv_data":82228312066362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555451523835,"port_rcv_data":2276845294047,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:47.110","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.08,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:47.124","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:47.137","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.53,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:47.149","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:47.160","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.91,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:47.171","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:47.181","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:47.191","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.12,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259248.030249,"monotonic":2346085.47931718,"hcas":{"mlx5_0":{"port_xmit_data":82839077979572,"port_rcv_data":84436250483744,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438934760555,"port_rcv_data":3204168066451,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82898834015718,"port_rcv_data":82228378222553,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555451579449,"port_rcv_data":2276898918851,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:48.147","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.36,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:48.161","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.29,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:48.175","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.01,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:48.187","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:48.198","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.87,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:48.209","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:48.220","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:48.230","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":145.83,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259249.0310404,"monotonic":2346086.480108601,"hcas":{"mlx5_0":{"port_xmit_data":82839090655152,"port_rcv_data":84437340731731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438934946508,"port_rcv_data":3204206146731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82899887374002,"port_rcv_data":82228446774266,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555452074411,"port_rcv_data":2276954957633,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:49.131","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:49.144","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:49.157","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.82,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:49.169","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:49.180","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:49.191","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:49.201","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.23,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:49.211","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.84,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259250.0318525,"monotonic":2346087.48092045,"hcas":{"mlx5_0":{"port_xmit_data":82839103611046,"port_rcv_data":84438447681335,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438935424063,"port_rcv_data":3204261368337,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82900940410456,"port_rcv_data":82228497470045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555452257636,"port_rcv_data":2276993140767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:50.112","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:50.125","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:50.139","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.59,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:50.151","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:50.162","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:50.172","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:50.183","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:50.192","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.38,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259251.0326824,"monotonic":2346088.481750393,"hcas":{"mlx5_0":{"port_xmit_data":82839116025628,"port_rcv_data":84439516639195,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438935480964,"port_rcv_data":3204291492991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82901979809674,"port_rcv_data":82228539070216,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555452287830,"port_rcv_data":2277022344219,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:51.130","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.22,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:51.144","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:51.158","gpu_util_pct":98.0,"memory_used_mib":79943.0,"power_w":147.11,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:51.170","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":146.41,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:51.182","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:51.193","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:51.203","gpu_util_pct":99.0,"memory_used_mib":79943.0,"power_w":147.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:51.213","gpu_util_pct":98.0,"memory_used_mib":80005.0,"power_w":140.96,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259252.0334897,"monotonic":2346089.482557538,"hcas":{"mlx5_0":{"port_xmit_data":82839127950955,"port_rcv_data":84440696783991,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438935511003,"port_rcv_data":3204320696374,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82903128463513,"port_rcv_data":82228580145487,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555452317823,"port_rcv_data":2277051547602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:52.130","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:52.144","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.85,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:52.157","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.77,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:52.170","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.37,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:52.181","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":149.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:52.192","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.26,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:52.203","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:52.213","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":146.64,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259253.0343018,"monotonic":2346090.483369682,"hcas":{"mlx5_0":{"port_xmit_data":82839140068591,"port_rcv_data":84441884755064,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438935698320,"port_rcv_data":3204344381677,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82904294068319,"port_rcv_data":82228624628575,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555452717506,"port_rcv_data":2277084074505,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:53.115","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:53.129","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:53.142","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.66,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:53.155","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":145.21,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:53.166","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:53.177","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":157.01,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:53.187","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:53.197","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":146.73,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259254.0349576,"monotonic":2346091.484025604,"hcas":{"mlx5_0":{"port_xmit_data":82839152355102,"port_rcv_data":84443170012557,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438935965649,"port_rcv_data":3204455244175,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82905469945501,"port_rcv_data":82228756435364,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555453151134,"port_rcv_data":2277203777983,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:54.135","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":159.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:54.149","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.75,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:54.163","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.04,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:54.175","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:54.186","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:54.197","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.75,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:54.207","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:54.217","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.41,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259255.0357661,"monotonic":2346092.484833767,"hcas":{"mlx5_0":{"port_xmit_data":82839164814008,"port_rcv_data":84444297354759,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438936285972,"port_rcv_data":3204472927591,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82906579925200,"port_rcv_data":82228777341932,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555453317418,"port_rcv_data":2277212566990,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:55.134","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:55.148","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.0,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:55.161","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.14,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:55.174","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.66,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:55.186","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:55.197","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:55.208","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:55.217","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.89,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259256.0365708,"monotonic":2346093.48563885,"hcas":{"mlx5_0":{"port_xmit_data":82839177437165,"port_rcv_data":84445379345979,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438936454720,"port_rcv_data":3204505283567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82907629097773,"port_rcv_data":82228839999802,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555453814983,"port_rcv_data":2277262866811,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:56.135","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:56.149","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.6,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:56.163","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.24,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:56.175","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:56.187","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:56.198","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":154.57,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:56.208","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:56.218","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.55,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259257.0373807,"monotonic":2346094.486448555,"hcas":{"mlx5_0":{"port_xmit_data":82839190367262,"port_rcv_data":84446466814475,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438936935980,"port_rcv_data":3204546377331,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82908676802909,"port_rcv_data":82228866827348,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555453829429,"port_rcv_data":2277277226767,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:57.118","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":151.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:57.132","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:57.145","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.1,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:57.158","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:57.169","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.26,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:57.180","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:57.190","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.78,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:57.200","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.46,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259258.038209,"monotonic":2346095.487276885,"hcas":{"mlx5_0":{"port_xmit_data":82839202685795,"port_rcv_data":84447627635025,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438937102155,"port_rcv_data":3204569786491,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82909814116306,"port_rcv_data":82228920490947,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555454308209,"port_rcv_data":2277318732577,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:58.155","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:58.169","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:58.183","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.8,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:58.195","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.7,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:58.206","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:58.217","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.61,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:58.228","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:58.237","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.38,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259259.0389993,"monotonic":2346096.48806705,"hcas":{"mlx5_0":{"port_xmit_data":82839215104931,"port_rcv_data":84448835034718,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438937575060,"port_rcv_data":3204611085654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82910978760334,"port_rcv_data":82228955852009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555454476368,"port_rcv_data":2277342141082,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:40:59.120","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:40:59.133","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:40:59.147","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.08,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:40:59.159","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:40:59.170","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":147.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:40:59.181","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.69,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:40:59.192","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.66,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:40:59.202","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259260.039156,"monotonic":2346097.488223715,"hcas":{"mlx5_0":{"port_xmit_data":82839227143388,"port_rcv_data":84450021046812,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438937589429,"port_rcv_data":3204625652567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82912152307997,"port_rcv_data":82228982265938,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555454490349,"port_rcv_data":2277356500831,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:00.139","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.3,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:00.153","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:00.166","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.2,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:00.178","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.5,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:00.189","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:00.200","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:00.210","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:00.220","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.11,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259261.0399642,"monotonic":2346098.489032068,"hcas":{"mlx5_0":{"port_xmit_data":82839239172461,"port_rcv_data":84451208543045,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438937603441,"port_rcv_data":3204640012247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82913323194522,"port_rcv_data":82229008629084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555454504361,"port_rcv_data":2277370860511,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:01.121","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:01.135","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:01.148","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.88,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:01.161","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:01.172","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.92,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:01.183","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:01.193","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:01.203","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.68,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259262.0407915,"monotonic":2346099.489859394,"hcas":{"mlx5_0":{"port_xmit_data":82839251460998,"port_rcv_data":84452434229247,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438937814551,"port_rcv_data":3204685355416,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82914507221279,"port_rcv_data":82229087630222,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455080465,"port_rcv_data":2277437608268,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:02.176","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":152.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:02.191","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.67,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:02.204","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.16,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:02.216","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:02.228","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:02.238","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:02.249","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:02.259","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":140.99,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259263.0415595,"monotonic":2346100.490627189,"hcas":{"mlx5_0":{"port_xmit_data":82839264043606,"port_rcv_data":84453701282723,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938346635,"port_rcv_data":3204775870760,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82915682196421,"port_rcv_data":82229168762303,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455306564,"port_rcv_data":2277506751223,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:03.176","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:03.190","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.62,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:03.203","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.69,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:03.215","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:03.226","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":147.38,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:03.237","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.21,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:03.247","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.98,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:03.257","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.8,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259264.0423353,"monotonic":2346101.491403088,"hcas":{"mlx5_0":{"port_xmit_data":82839276182814,"port_rcv_data":84454879077819,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938499976,"port_rcv_data":3204784583166,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82916850365439,"port_rcv_data":82229207448493,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455772292,"port_rcv_data":2277533507764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:04.143","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.9,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:04.157","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:04.170","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.15,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:04.182","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:04.193","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.59,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:04.204","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.58,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:04.214","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.99,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:04.224","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259265.043137,"monotonic":2346102.492204831,"hcas":{"mlx5_0":{"port_xmit_data":82839288684721,"port_rcv_data":84456083750441,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948546,"port_rcv_data":3204811340362,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82918030584080,"port_rcv_data":82229228441944,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922286,"port_rcv_data":2277542426818,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:05.142","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:05.156","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.93,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:05.169","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.86,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:05.181","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.55,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:05.192","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:05.203","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:05.213","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.3,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:05.223","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.99,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259266.0439408,"monotonic":2346103.493008905,"hcas":{"mlx5_0":{"port_xmit_data":82839300317948,"port_rcv_data":84457276681712,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82919223428386,"port_rcv_data":82229240085525,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:06.125","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:06.138","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:06.152","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.66,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:06.164","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:06.176","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:06.187","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.7,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:06.197","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":150.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:06.208","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.97,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259267.044759,"monotonic":2346104.493827036,"hcas":{"mlx5_0":{"port_xmit_data":82839311678570,"port_rcv_data":84458502121280,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82920446659129,"port_rcv_data":82229251435170,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:07.143","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:07.157","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:07.171","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.34,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:07.183","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:07.195","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.48,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:07.205","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.97,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:07.216","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.65,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:07.225","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.0,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259268.045559,"monotonic":2346105.49462707,"hcas":{"mlx5_0":{"port_xmit_data":82839323086452,"port_rcv_data":84459733625038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82921678170231,"port_rcv_data":82229262861943,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:08.126","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:08.139","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":147.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:08.153","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.03,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:08.165","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:08.176","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.42,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:08.187","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:08.197","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:08.207","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.11,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259269.0463789,"monotonic":2346106.495446908,"hcas":{"mlx5_0":{"port_xmit_data":82839334620087,"port_rcv_data":84460944418644,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82922889729683,"port_rcv_data":82229274424702,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:09.126","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.25,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:09.140","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:09.153","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.14,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:09.165","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.24,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:09.176","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:09.187","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:09.197","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.07,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:09.207","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.88,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259270.0471992,"monotonic":2346107.496267339,"hcas":{"mlx5_0":{"port_xmit_data":82839346701154,"port_rcv_data":84462124675191,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82924069225744,"port_rcv_data":82229286501474,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:10.128","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:10.141","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.28,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:10.155","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.2,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:10.167","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:10.179","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.77,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:10.190","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.8,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:10.200","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.78,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:10.210","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.64,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259271.0480177,"monotonic":2346108.497085726,"hcas":{"mlx5_0":{"port_xmit_data":82839358633300,"port_rcv_data":84463288966121,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82925234881230,"port_rcv_data":82229298460649,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:11.148","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:11.162","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.61,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:11.175","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.39,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:11.187","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:11.199","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.24,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:11.210","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.94,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:11.220","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:11.230","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":142.34,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259272.048817,"monotonic":2346109.497884902,"hcas":{"mlx5_0":{"port_xmit_data":82839370626540,"port_rcv_data":84464457553567,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82926404622116,"port_rcv_data":82229310492288,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:12.130","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.64,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:12.144","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":148.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:12.157","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.01,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:12.170","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:12.181","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:12.193","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:12.203","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.42,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:12.214","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.79,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259273.0496328,"monotonic":2346110.498700853,"hcas":{"mlx5_0":{"port_xmit_data":82839382678135,"port_rcv_data":84465633547669,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82927578111354,"port_rcv_data":82229322522292,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:13.146","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.47,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:13.160","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.82,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:13.173","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.57,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:13.185","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":140.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:13.197","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:13.208","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.37,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:13.218","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.62,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:13.228","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.91,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259274.0504394,"monotonic":2346111.499507272,"hcas":{"mlx5_0":{"port_xmit_data":82839394662640,"port_rcv_data":84466808223403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82928754801062,"port_rcv_data":82229334555029,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:14.149","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":153.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:14.163","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.13,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:14.176","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.78,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:14.189","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":141.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:14.201","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":146.95,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:14.212","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":153.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:14.222","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.85,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:14.232","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":144.71,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259275.0512378,"monotonic":2346112.50030594,"hcas":{"mlx5_0":{"port_xmit_data":82839406690562,"port_rcv_data":84467978981414,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82929925574638,"port_rcv_data":82229346585084,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:15.132","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":155.33,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:15.146","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:15.159","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.7,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:15.171","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:15.183","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.44,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:15.194","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:15.204","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":149.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:15.214","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.85,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259276.0520575,"monotonic":2346113.501125525,"hcas":{"mlx5_0":{"port_xmit_data":82839418640192,"port_rcv_data":84469145452307,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82931091714918,"port_rcv_data":82229358546896,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:16.133","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:16.147","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":150.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:16.160","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.38,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:16.172","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":144.63,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:16.183","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.0,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:16.194","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.96,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:16.204","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":148.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:16.214","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":143.52,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259277.0528767,"monotonic":2346114.501944956,"hcas":{"mlx5_0":{"port_xmit_data":82839430671960,"port_rcv_data":84470320982375,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82932267205106,"port_rcv_data":82229370584716,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:17.152","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":154.86,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:17.165","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:17.178","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.82,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:17.190","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.52,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:17.201","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":147.79,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:17.212","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.29,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:17.223","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":147.79,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:17.232","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":145.12,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259278.053688,"monotonic":2346115.502755838,"hcas":{"mlx5_0":{"port_xmit_data":82839442588279,"port_rcv_data":84471482889935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82933429470905,"port_rcv_data":82229382521459,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:18.188","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":158.1,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:18.202","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:18.215","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.55,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:18.229","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:18.240","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":148.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:18.251","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":155.48,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:18.262","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":151.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:18.272","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":146.28,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259279.0544474,"monotonic":2346116.503515179,"hcas":{"mlx5_0":{"port_xmit_data":82839454830628,"port_rcv_data":84472577194431,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82934523574019,"port_rcv_data":82229394787600,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:19.155","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":156.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:19.168","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":151.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:19.181","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.17,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:19.194","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":143.11,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:19.205","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:19.216","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:19.226","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":146.16,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:19.236","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":141.48,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259280.0552516,"monotonic":2346117.504319512,"hcas":{"mlx5_0":{"port_xmit_data":82839467359623,"port_rcv_data":84473592032927,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82935542805135,"port_rcv_data":82229407340826,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:20.154","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:20.168","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":149.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:20.182","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.56,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:20.195","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":142.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:20.206","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.98,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:20.218","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":152.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:20.228","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.01,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:20.239","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.72,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259281.0560489,"monotonic":2346118.505116776,"hcas":{"mlx5_0":{"port_xmit_data":82839480902912,"port_rcv_data":84474560938788,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82936505529849,"port_rcv_data":82229420875102,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:21.155","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":143.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:21.169","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:21.182","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":138.3,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:21.194","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.08,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:21.205","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:21.216","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.53,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:21.226","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:21.236","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.32,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259282.056853,"monotonic":2346119.505920893,"hcas":{"mlx5_0":{"port_xmit_data":82839494260245,"port_rcv_data":84475516703960,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82937463246204,"port_rcv_data":82229434299346,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:22.138","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:22.152","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:22.166","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.49,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:22.178","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:22.189","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.55,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:22.200","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:22.210","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:22.220","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":139.25,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259283.057672,"monotonic":2346120.506739994,"hcas":{"mlx5_0":{"port_xmit_data":82839507669700,"port_rcv_data":84476475390685,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82938419996381,"port_rcv_data":82229447692594,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:23.139","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.43,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:23.153","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:23.166","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.89,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:23.179","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":139.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:23.190","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":139.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:23.201","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:23.212","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":143.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:23.222","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.6,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259284.0584896,"monotonic":2346121.507557476,"hcas":{"mlx5_0":{"port_xmit_data":82839521008272,"port_rcv_data":84477429002213,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82939375866392,"port_rcv_data":82229461089875,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:24.139","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.77,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:24.153","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:24.167","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.16,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:24.179","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:24.190","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":145.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:24.201","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.04,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:24.212","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:24.222","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.79,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259285.059307,"monotonic":2346122.508375125,"hcas":{"mlx5_0":{"port_xmit_data":82839534393221,"port_rcv_data":84478386207002,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82940333103154,"port_rcv_data":82229474501078,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:25.159","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.81,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:25.173","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.06,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:25.187","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.28,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:25.199","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:25.211","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":144.28,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:25.222","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:25.232","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.83,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:25.242","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.14,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259286.0601065,"monotonic":2346123.509174496,"hcas":{"mlx5_0":{"port_xmit_data":82839547855115,"port_rcv_data":84479350216739,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82941296925570,"port_rcv_data":82229487999202,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:26.160","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":150.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:26.174","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:26.187","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.59,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:26.199","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.43,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:26.211","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":142.35,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:26.222","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.18,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:26.233","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":145.58,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:26.243","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":138.6,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259287.0609074,"monotonic":2346124.509975336,"hcas":{"mlx5_0":{"port_xmit_data":82839561224420,"port_rcv_data":84480305853778,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82942251730934,"port_rcv_data":82229501374470,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:27.176","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:27.190","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.31,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:27.203","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.17,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:27.216","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":138.89,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:27.228","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":140.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:27.239","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":146.88,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:27.249","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:27.259","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":135.53,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259288.0616899,"monotonic":2346125.510757701,"hcas":{"mlx5_0":{"port_xmit_data":82839574701489,"port_rcv_data":84481245292727,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82943191633271,"port_rcv_data":82229514881518,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:28.160","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:28.174","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":141.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:28.188","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.69,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:28.200","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":133.87,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:28.211","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":141.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:28.223","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.63,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:28.233","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":144.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:28.243","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":137.24,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259289.062493,"monotonic":2346126.511561261,"hcas":{"mlx5_0":{"port_xmit_data":82839588964596,"port_rcv_data":84482045184074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82943991376301,"port_rcv_data":82229529143792,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:29.162","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:29.176","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":140.37,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:29.189","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":137.49,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:29.201","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:29.213","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:29.223","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":144.39,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:29.233","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.68,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:29.243","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.58,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259290.0632966,"monotonic":2346127.512364598,"hcas":{"mlx5_0":{"port_xmit_data":82839603243162,"port_rcv_data":84482847199554,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82944793597775,"port_rcv_data":82229543439798,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:30.164","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.76,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:30.178","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":138.71,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:30.191","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.88,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:30.204","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:30.215","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.01,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:30.226","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:30.236","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":141.37,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:30.246","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.49,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259291.0640805,"monotonic":2346128.513148551,"hcas":{"mlx5_0":{"port_xmit_data":82839617489807,"port_rcv_data":84483647248397,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82945593883314,"port_rcv_data":82229557707659,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:31.163","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":145.71,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:31.176","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.13,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:31.190","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":134.63,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:31.203","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":136.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:31.215","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":137.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:31.226","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":145.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:31.237","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":142.84,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:31.247","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.6,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259292.0648634,"monotonic":2346129.513931587,"hcas":{"mlx5_0":{"port_xmit_data":82839631816967,"port_rcv_data":84484452149455,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82946397420583,"port_rcv_data":82229572019507,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:32.162","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":146.99,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:32.176","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.44,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:32.190","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":136.83,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:32.203","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":137.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:32.214","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.75,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:32.226","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":142.87,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:32.236","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":140.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:32.246","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":134.23,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259293.065655,"monotonic":2346130.514722952,"hcas":{"mlx5_0":{"port_xmit_data":82839646039218,"port_rcv_data":84485250735843,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82947197121009,"port_rcv_data":82229586275677,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:33.146","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":147.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:33.160","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.68,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:33.173","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.16,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:33.186","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":135.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:33.197","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":138.69,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:33.208","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":143.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:33.218","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":139.66,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:33.228","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":136.0,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259294.0664585,"monotonic":2346131.51552631,"hcas":{"mlx5_0":{"port_xmit_data":82839659479072,"port_rcv_data":84485849815978,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82947795802226,"port_rcv_data":82229599704777,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:34.184","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.38,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:34.198","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":133.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:34.211","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.09,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:34.224","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":130.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:34.235","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":134.76,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:34.247","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.6,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:34.257","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":133.82,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:34.267","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.19,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259295.0672233,"monotonic":2346132.516291409,"hcas":{"mlx5_0":{"port_xmit_data":82839672957937,"port_rcv_data":84486444400678,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82948388909486,"port_rcv_data":82229613134997,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:35.167","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.55,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:35.181","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.88,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:35.193","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.5,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:35.206","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.5,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:35.217","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":136.04,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:35.228","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.82,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:35.238","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.55,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:35.247","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":131.29,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259296.0680108,"monotonic":2346133.51707869,"hcas":{"mlx5_0":{"port_xmit_data":82839686339926,"port_rcv_data":84487033632900,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82948979718029,"port_rcv_data":82229626550063,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:36.204","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":141.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:36.217","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":136.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:36.230","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":128.13,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:36.242","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":129.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:36.253","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":131.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:36.264","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":139.11,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:36.274","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.62,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:36.283","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":129.39,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259297.0687678,"monotonic":2346134.517835707,"hcas":{"mlx5_0":{"port_xmit_data":82839699718529,"port_rcv_data":84487622864627,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82949568733227,"port_rcv_data":82229639926697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:37.167","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":140.8,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:37.180","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":134.47,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:37.193","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":130.72,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:37.204","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":131.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:37.214","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":132.6,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:37.225","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":135.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:37.234","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":135.44,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:37.243","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":128.1,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259298.0695817,"monotonic":2346135.518649635,"hcas":{"mlx5_0":{"port_xmit_data":82839707506467,"port_rcv_data":84487979952214,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82949924382694,"port_rcv_data":82229647661696,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:38.166","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":135.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:38.180","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":124.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:38.193","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":123.56,"sm_clock_mhz":2257.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:38.205","gpu_util_pct":100.0,"memory_used_mib":79963.0,"power_w":124.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:38.216","gpu_util_pct":100.0,"memory_used_mib":79875.0,"power_w":125.37,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:38.227","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":128.52,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:38.237","gpu_util_pct":100.0,"memory_used_mib":79943.0,"power_w":127.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:38.247","gpu_util_pct":100.0,"memory_used_mib":80005.0,"power_w":122.7,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259299.0703917,"monotonic":2346136.519459481,"hcas":{"mlx5_0":{"port_xmit_data":82839714143143,"port_rcv_data":84488289328752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82950233733782,"port_rcv_data":82229654283756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:39.213","gpu_util_pct":100.0,"memory_used_mib":79997.0,"power_w":127.67,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:39.230","gpu_util_pct":100.0,"memory_used_mib":79959.0,"power_w":122.25,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:39.248","gpu_util_pct":61.0,"memory_used_mib":79943.0,"power_w":114.09,"sm_clock_mhz":2265.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:39.264","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":115.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:39.279","gpu_util_pct":74.0,"memory_used_mib":79875.0,"power_w":120.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:39.293","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":119.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:39.307","gpu_util_pct":15.0,"memory_used_mib":79943.0,"power_w":117.09,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:39.320","gpu_util_pct":0.0,"memory_used_mib":80005.0,"power_w":115.9,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259300.0711172,"monotonic":2346137.520184874,"hcas":{"mlx5_0":{"port_xmit_data":82839714143143,"port_rcv_data":84488289328752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82950233733782,"port_rcv_data":82229654283756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:40.194","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":80.9,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:40.212","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":83.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:40.230","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":80.61,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:40.246","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":84.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:40.261","gpu_util_pct":0.0,"memory_used_mib":79875.0,"power_w":87.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:40.276","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":85.28,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:40.290","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":85.91,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:40.303","gpu_util_pct":0.0,"memory_used_mib":80005.0,"power_w":83.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259301.0718625,"monotonic":2346138.520930508,"hcas":{"mlx5_0":{"port_xmit_data":82839714143143,"port_rcv_data":84488289328752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82950233733782,"port_rcv_data":82229654283756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:41.173","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":79.97,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:41.191","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":82.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:41.208","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":78.76,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:41.224","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":79.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:41.239","gpu_util_pct":0.0,"memory_used_mib":79875.0,"power_w":80.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:41.254","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:41.268","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.19,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:41.281","gpu_util_pct":0.0,"memory_used_mib":80005.0,"power_w":79.58,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259302.0726311,"monotonic":2346139.521699157,"hcas":{"mlx5_0":{"port_xmit_data":82839714143143,"port_rcv_data":84488289328752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82950233733782,"port_rcv_data":82229654283756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:42.209","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":79.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:42.227","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":82.86,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:42.244","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":79.25,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:42.260","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":79.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:42.274","gpu_util_pct":0.0,"memory_used_mib":79875.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:42.289","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":81.05,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:42.302","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.59,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:42.316","gpu_util_pct":0.0,"memory_used_mib":80005.0,"power_w":79.34,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259303.0733638,"monotonic":2346140.52243156,"hcas":{"mlx5_0":{"port_xmit_data":82839714143143,"port_rcv_data":84488289328752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82950233733782,"port_rcv_data":82229654283756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:43.217","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":79.64,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:43.236","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":82.61,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:43.252","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":79.0,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:43.268","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":79.44,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:43.283","gpu_util_pct":0.0,"memory_used_mib":79875.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:43.297","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.54,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:43.311","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":81.1,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:43.324","gpu_util_pct":0.0,"memory_used_mib":80005.0,"power_w":79.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788259304.0740912,"monotonic":2346141.523159189,"hcas":{"mlx5_0":{"port_xmit_data":82839714143143,"port_rcv_data":84488289328752,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":3438938948996,"port_rcv_data":3204811547596,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":82950233733782,"port_rcv_data":82229654283756,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2555455922317,"port_rcv_data":2277542426887,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/01 18:41:44.193","gpu_util_pct":0.0,"memory_used_mib":79997.0,"power_w":79.88,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/01 18:41:44.208","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":82.12,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/01 18:41:44.223","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":78.75,"sm_clock_mhz":2272.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/01 18:41:44.238","gpu_util_pct":0.0,"memory_used_mib":79963.0,"power_w":79.68,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/01 18:41:44.253","gpu_util_pct":0.0,"memory_used_mib":79875.0,"power_w":80.46,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/01 18:41:44.267","gpu_util_pct":0.0,"memory_used_mib":79959.0,"power_w":80.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/01 18:41:44.281","gpu_util_pct":0.0,"memory_used_mib":79943.0,"power_w":80.61,"sm_clock_mhz":2400.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/01 18:41:44.295","gpu_util_pct":0.0,"memory_used_mib":80005.0,"power_w":79.58,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0}]}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/bench/smoke.json

@@ -0,0 +1 @@
+{"text":" 5\nQuestion: A train travels 60 miles in 1 hour.","output_ids":[220,20,198,29328,25,401,9010,41333,220,2205,12034,306,220,16,9185,13],"meta_info":{"id":"5666cdd4862b4c3cb6b6b793ff54a64c","finish_reason":{"type":"length","length":16},"prompt_tokens":22,"weight_version":"default","num_retractions":0,"forward_entry_time":1788315330.1490085,"queue_time":0.00040340423583984375,"reasoning_tokens":0,"completion_tokens":16,"cached_tokens":0,"weight_versions":[{"version":"default","start":0,"end":16}],"cached_tokens_details":null,"dp_rank":null,"e2e_latency":3.3326520258560777,"spec_accept_rate":0.10476190476190476,"spec_accept_length":2.2857142857142856,"spec_num_correct_drafts":11,"spec_num_proposed_drafts":105,"spec_verify_ct":7,"spec_accepted_drafts":11,"spec_proposed_drafts":105,"spec_correct_drafts_histogram":[2,2,2,0,0,1],"spec_accept_histogram":[2,2,2,0,0,1],"request_received_ts":1788315327.5102177,"api_server_dispatch_finish_ts":1788315327.5115225,"request_finished_ts":1788315330.8428695,"decode_throughput":3164579.4989684643,"response_sent_to_client_ts":1788315330.8435922}}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/benchmark.cmd.txt

@@ -0,0 +1 @@
+docker run --rm --network host -v /data:/data -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.benchmark.serving --backend sglang --host 174.1.60.1 --port 31000 --model /data/hf_models/Kimi-K3 --tokenizer /data/hf_models/Kimi-K3 --dataset-name speed-bench --dataset-path /data/speedbench16k_complete.jsonl --num-prompts 8 --max-concurrency 8 --request-rate 10000 --speed-bench-output-len 512 --temperature 0.0 --top-p 1.0 --seed 42 --warmup-requests 1 --output-details --output-file /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/bench/dflash_speedbench16k_c8_o512.jsonl 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/d_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_d_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.5 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint nsys local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 profile --trace=cuda\,nvtx\,nccl --nccl-trace=api\,group\,gpu\,coll\,kernel-launch --sample=none --cpuctxsw=none --capture-range=cudaProfilerApi --capture-range-end=stop --cuda-graph-trace=node --force-overwrite=true --stats=true --output /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/nsys/d0 python3 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/d_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_d_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.6 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/d_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_d_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.7 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/d_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_d_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.8 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 32 --pp-size 1 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.5:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.86 --mamba-full-memory-ratio 0.21 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode decode --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/p_0.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_p_0 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.1 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 0 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/p_1.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_p_1 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.2 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 1 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/p_2.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_p_2 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.3 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 2 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/p_3.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_p_3 --gpus all --network host --ipc=host --ulimit memlock=-1 --device /dev/infiniband -v /data/hf_models/Kimi-K3:/data/hf_models/Kimi-K3:ro -v /data/hzy/cache/kimi-dflash-fi0618-situ4460:/cache -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251:/data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251 -e SGLANG_HOST_IP=174.1.60.4 -e PYTHONUNBUFFERED=1 -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 -e GLOO_SOCKET_IFNAME=bond0 -e NCCL_SOCKET_IFNAME=bond1 -e NCCL_IB_HCA=mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 -e NCCL_IB_GID_INDEX=3 -e NCCL_IB_TIMEOUT=22 -e NCCL_IB_RETRY_CNT=7 -e NCCL_CUMEM_ENABLE=1 -e SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK=0 -e SGLANG_MOE_FUSED_GATE_RADIX=1 -e FLASHINFER_WORKSPACE_BASE=/cache -e FLASHINFER_CUDA_ARCH_LIST=12.0f -e SGLANG_CACHE_DIR=/cache -e XDG_CACHE_HOME=/cache -e TRITON_CACHE_DIR=/cache/triton -e TORCH_EXTENSIONS_DIR=/cache/torch_extensions -e CUDA_CACHE_PATH=/cache/cuda -e TMPDIR=/cache/tmp -e MAX_JOBS=4 -v /data/hf_models/Kimi-K3-DFlash:/data/hf_models/Kimi-K3-DFlash:ro -v /data/hzy/kimi3_pd_dflash_attribution/kimi3-pd-dflash-nsys-donly-20260902-100251/assets/dflash_worker_v2_profile.py:/opt/kimi-dflash/python/sglang/srt/speculative/dflash_worker_v2.py:ro --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang.launch_server --model-path /data/hf_models/Kimi-K3 --served-model-name kimi-k3 --trust-remote-code --tp-size 4 --pp-size 8 --ep-size 4 --nnodes 4 --node-rank 3 --dist-init-addr 174.1.60.1:20000 --moe-runner-backend flashinfer_mxfp4 --moe-a2a-backend none --kv-cache-dtype bfloat16 --chunked-prefill-size 8192 --page-size 64 --mem-fraction-static 0.84 --mamba-full-memory-ratio 0.36 --mamba-radix-cache-strategy extra_buffer_lazy --disable-radix-cache --max-running-requests 8 --cuda-graph-max-bs-decode 8 --dist-timeout 3600 --disaggregation-transfer-backend mooncake --disaggregation-mode prefill --disaggregation-bootstrap-port 28800 --disaggregation-ib-device mlx5_0\,mlx5_1\,mlx5_2\,mlx5_3 --enable-metrics --enable-request-time-stats-logging --host 0.0.0.0 --port 30000 --speculative-draft-kv-cache-dtype bfloat16 --speculative-algorithm DFLASH --speculative-draft-model-path /data/hf_models/Kimi-K3-DFlash --speculative-num-draft-tokens 16 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/commands/router.cmd.txt

@@ -0,0 +1 @@
+docker run -d --name kimi3_pro6000_pd_dflash_attribution_dflash_nsys_router_0 --network host --entrypoint python3 local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1 -m sglang_router.launch_router --pd-disaggregation --mini-lb --prefill http://174.1.60.1:30000 28800 --decode http://174.1.60.5:30000 --host 0.0.0.0 --port 31000 

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/metadata/input_sha256.txt

@@ -0,0 +1,5 @@
+2af605e738d3a9b3f17310d54a81e81f1342b76eeeb12da7cd3f65abf6e0764f  /data/speedbench16k_complete.jsonl
+fdf4a953c93857572b6dd62eca4518e9f7d76b5c47f0397e0a915a5915e5701a  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/dflash_worker_v2_profile.py
+bc0b042e0290296da4f003b5d1a859a952d9bc0a497589bd948dc0b281fbf1b7  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/profile_tools.py
+1d0b77b212dbf2541f023844c9e42a8f97b754917738accc3742c1b5119a4594  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/run_pd_dflash_attribution.sh
+5eda3098a6460d8f7450cc0334bbb9cc919ae88d02804baa2ce87cabec7850e7  /data/hzy/sskj/experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/config.env

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/metadata/manifest.env

@@ -0,0 +1,7 @@
+RUN_ID=kimi3-pd-dflash-nsys-donly-20260902-100251
+RUN_MODE=dflash
+PROFILE_KIND=nsys
+IMAGE=local/sglang:kimi-k3-pp-dflash-33863-fi0618-situ4460-kvbounds1
+CHUNK=8192
+OUTPUT_LEN=512
+CONCURRENCY=8

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/metadata/preflight.txt

@@ -0,0 +1,8 @@
+174.1.60.1 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.2 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.3 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.4 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.5 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.6 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.7 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f
+174.1.60.8 sha256:7aa2be2d7af6cb11fea6d0c00d9bcd323ec986095dc9c4d3c997cff15ccb3b3f

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/metadata/profile_arm_d.json

@@ -0,0 +1 @@
+Start profiling.

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/metadata/profile_arm_p.json

@@ -0,0 +1 @@
+Start profiling.

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/metadata/profile_stop_d.json

@@ -0,0 +1 @@
+Internal Server Error

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/metadata/profile_stop_p.json

@@ -0,0 +1 @@
+Internal Server Error

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/nsys_nvtx_kern_sum.txt

@@ -0,0 +1,948 @@
+
+NOTICE: Existing SQLite export found: /report/profiles/node_5/nsys/d0.sqlite
+        It is assumed file was previously exported from: /report/profiles/node_5/nsys/d0.nsys-rep
+        Consider using --force-export=true if needed.
+
+Processing [/report/profiles/node_5/nsys/d0.sqlite] with [/opt/nvidia/nsight-systems-cli/2026.4.1/target-linux-x64/reports/nvtx_kern_sum.py]... 
+
+ ** NVTX Range Kernel Summary (nvtx_kern_sum):
+
+ NVTX Range  Style  PID  TID  NVTX Inst  Kern Inst  Total Time (ns)  Avg (ns)  Med (ns)  Min (ns)  Max (ns)  StdDev (ns)                                              Kernel Name                                             
+ ----------  -----  ---  ---  ---------  ---------  ---------------  --------  --------  --------  --------  -----------  ----------------------------------------------------------------------------------------------------
+                    261  261          0         20          313,034  15,651.7  15,584.5    15,265    16,864        356.0  create_flashinfer_kv_indices_triton                                                                 
+                    261  261          0         40          231,910   5,797.8   2,432.0     1,312    11,264      4,063.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    261  261          0         42           97,571   2,323.1   2,288.0     1,312     2,912        336.8  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    261  261          0         10           43,682   4,368.2   4,912.0     1,408     5,568      1,519.0  alloc_extend_kernel                                                                                 
+                    261  261          0         36           39,618   1,100.5   1,104.5       736     1,600        211.8  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    261  261          0         23           34,784   1,512.3   1,440.0     1,408     2,176        177.2  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    261  261          0         20           30,240   1,512.0   1,504.0     1,472     1,568         34.2  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    261  261          0         20           23,489   1,174.5   1,184.0     1,121     1,312         37.5  _fused_replay_state_indices_kernel                                                                  
+                    261  261          0         20           21,472   1,073.6   1,056.0     1,056     1,184         30.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    261  261          0         20           20,992   1,049.6   1,056.0     1,024     1,184         35.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    261  261          0         20           14,209     710.5     704.0       672       800         36.8  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    261  261          0          8           12,897   1,612.1   1,600.0     1,536     1,792         88.7  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    261  261          0          8           10,018   1,252.3   1,120.0     1,089     2,240        399.4  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    261  261          0          2            2,849   1,424.5   1,424.5     1,344     1,505        113.8  assign_req_to_token_pool                                                                            
+                    261  261          0          2            2,656   1,328.0   1,328.0     1,216     1,440        158.4  get_last_loc_kernel                                                                                 
+                    261  261          0          1            2,112   2,112.0   2,112.0     2,112     2,112          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    261  261          0          1            1,152   1,152.0   1,152.0     1,152     1,152          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+                    262  262          0         19          287,718  15,143.1  15,136.0    14,848    15,809        231.0  create_flashinfer_kv_indices_triton                                                                 
+                    262  262          0         40          156,423   3,910.6   2,432.5     1,312     7,937      2,071.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    262  262          0         40           93,028   2,325.7   2,368.0     1,344     3,009        314.5  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    262  262          0         10           43,266   4,326.6   4,816.5     1,377     5,728      1,511.9  alloc_extend_kernel                                                                                 
+                    262  262          0         35           38,305   1,094.4   1,088.0       768     1,600        211.1  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    262  262          0         22           32,353   1,470.6   1,408.0     1,376     2,112        177.3  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    262  262          0         19           28,576   1,504.0   1,472.0     1,408     1,632         69.1  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    262  262          0         19           22,753   1,197.5   1,184.0     1,152     1,312         37.4  _fused_replay_state_indices_kernel                                                                  
+                    262  262          0         19           19,552   1,029.1   1,024.0       992     1,152         34.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    262  262          0         19           16,096     847.2     832.0       832       864         16.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    262  262          0         19           13,089     688.9     672.0       672       768         27.0  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    262  262          0          8           12,416   1,552.0   1,536.0     1,472     1,696         80.2  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    262  262          0          8           10,049   1,256.1   1,120.0     1,088     2,272        410.7  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    262  262          0          2            2,912   1,456.0   1,456.0     1,280     1,632        248.9  assign_req_to_token_pool                                                                            
+                    262  262          0          2            2,624   1,312.0   1,312.0     1,184     1,440        181.0  get_last_loc_kernel                                                                                 
+                    262  262          0          1            2,112   2,112.0   2,112.0     2,112     2,112          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    262  262          0          1            1,152   1,152.0   1,152.0     1,152     1,152          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+                    263  263          0         19          289,769  15,251.0  15,265.0    14,912    15,681        198.3  create_flashinfer_kv_indices_triton                                                                 
+                    263  263          0         40          150,406   3,760.2   2,448.5     1,312     6,752      1,769.6  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    263  263          0         40           93,731   2,343.3   2,288.5     1,344     2,720        291.4  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    263  263          0         10           43,585   4,358.5   4,928.0     1,344     5,440      1,533.4  alloc_extend_kernel                                                                                 
+                    263  263          0         35           39,330   1,123.7   1,120.0       768     1,601        209.6  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    263  263          0         22           33,410   1,518.6   1,440.0     1,344     2,176        181.2  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    263  263          0         19           28,096   1,478.7   1,472.0     1,440     1,568         29.4  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    263  263          0         19           22,977   1,209.3   1,184.0     1,152     1,344         46.0  _fused_replay_state_indices_kernel                                                                  
+                    263  263          0         19           20,161   1,061.1   1,056.0       992     1,120         30.7  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    263  263          0         19           16,417     864.1     864.0       832       897         15.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    263  263          0         19           13,537     712.5     704.0       704       768         20.9  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    263  263          0          8           12,800   1,600.0   1,568.0     1,504     1,792        101.2  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    263  263          0          8           10,304   1,288.0   1,152.0     1,120     2,304        410.8  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    263  263          0          2            2,944   1,472.0   1,472.0     1,344     1,600        181.0  assign_req_to_token_pool                                                                            
+                    263  263          0          2            2,752   1,376.0   1,376.0     1,312     1,440         90.5  get_last_loc_kernel                                                                                 
+                    263  263          0          1            2,144   2,144.0   2,144.0     2,144     2,144          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    263  263          0          1            1,120   1,120.0   1,120.0     1,120     1,120          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+                    264  264          0         19          296,970  15,630.0  15,584.0    15,105    16,256        264.1  create_flashinfer_kv_indices_triton                                                                 
+                    264  264          0         40          223,749   5,593.7   2,528.0     1,344    12,000      3,720.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    264  264          0         40           94,594   2,364.8   2,240.0     1,408     2,912        332.4  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    264  264          0         10           43,841   4,384.1   4,928.0     1,408     5,568      1,502.1  alloc_extend_kernel                                                                                 
+                    264  264          0         35           39,072   1,116.3   1,120.0       768     1,600        209.2  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    264  264          0         22           33,440   1,520.0   1,456.0     1,376     2,176        177.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    264  264          0         19           28,352   1,492.2   1,504.0     1,408     1,600         48.0  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    264  264          0         19           22,656   1,192.4   1,184.0     1,152     1,280         27.9  _fused_replay_state_indices_kernel                                                                  
+                    264  264          0         19           20,225   1,064.5   1,056.0     1,024     1,152         25.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    264  264          0         19           16,640     875.8     864.0       864       896         15.9  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    264  264          0         19           13,664     719.2     704.0       704       832         36.0  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    264  264          0          8           12,737   1,592.1   1,568.0     1,504     1,761         78.2  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    264  264          0          8           10,240   1,280.0   1,136.0     1,120     2,304        414.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    264  264          0          2            2,880   1,440.0   1,440.0     1,344     1,536        135.8  assign_req_to_token_pool                                                                            
+                    264  264          0          2            2,688   1,344.0   1,344.0     1,280     1,408         90.5  get_last_loc_kernel                                                                                 
+                    264  264          0          1            2,080   2,080.0   2,080.0     2,080     2,080          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    264  264          0          1            1,184   1,184.0   1,184.0     1,184     1,184          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+                    265  265          0         19          291,720  15,353.7  15,361.0    14,976    15,936        282.7  create_flashinfer_kv_indices_triton                                                                 
+                    265  265          0         40          168,423   4,210.6   2,496.0     1,344     7,968      2,295.6  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    265  265          0         40           95,168   2,379.2   2,528.0     1,312     2,720        307.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    265  265          0         10           43,969   4,396.9   4,960.0     1,408     5,600      1,530.8  alloc_extend_kernel                                                                                 
+                    265  265          0         35           39,362   1,124.6   1,120.0       768     1,632        212.3  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    265  265          0         22           33,056   1,502.5   1,440.0     1,408     2,144        179.3  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    265  265          0         19           28,609   1,505.7   1,504.0     1,440     1,664         43.3  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    265  265          0         19           22,561   1,187.4   1,184.0     1,152     1,344         48.8  _fused_replay_state_indices_kernel                                                                  
+                    265  265          0         19           19,872   1,045.9   1,056.0     1,024     1,056         15.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    265  265          0         19           16,224     853.9     864.0       832       864         15.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    265  265          0         19           13,376     704.0     704.0       672       736         10.7  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    265  265          0          8           12,672   1,584.0   1,552.0     1,536     1,728         68.4  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    265  265          0          8           10,400   1,300.0   1,152.0     1,152     2,272        393.0  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    265  265          0          2            2,912   1,456.0   1,456.0     1,376     1,536        113.1  assign_req_to_token_pool                                                                            
+                    265  265          0          2            2,848   1,424.0   1,424.0     1,152     1,696        384.7  get_last_loc_kernel                                                                                 
+                    265  265          0          1            2,112   2,112.0   2,112.0     2,112     2,112          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    265  265          0          1            1,152   1,152.0   1,152.0     1,152     1,152          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+                    266  266          0         19          288,424  15,180.2  15,168.0    14,752    15,841        278.2  create_flashinfer_kv_indices_triton                                                                 
+                    266  266          0         40          162,342   4,058.6   2,400.5     1,280     7,937      2,129.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    266  266          0         40           92,098   2,302.4   2,176.0     1,312     2,752        324.2  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    266  266          0         10           43,712   4,371.2   4,912.0     1,344     5,664      1,514.8  alloc_extend_kernel                                                                                 
+                    266  266          0         35           38,850   1,110.0   1,088.0       768     1,601        221.6  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    266  266          0         22           32,129   1,460.4   1,376.0     1,344     2,176        194.9  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    266  266          0         19           28,224   1,485.5   1,472.0     1,440     1,632         43.1  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    266  266          0         19           21,952   1,155.4   1,152.0     1,120     1,344         54.3  _fused_replay_state_indices_kernel                                                                  
+                    266  266          0         19           19,618   1,032.5   1,024.0       992     1,344         77.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    266  266          0         19           16,001     842.2     832.0       832       865         15.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    266  266          0         19           12,930     680.5     672.0       640       800         39.7  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    266  266          0          8           12,578   1,572.3   1,536.5     1,504     1,728         91.1  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    266  266          0          8           10,081   1,260.1   1,120.0     1,120     2,240        395.9  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    266  266          0          2            2,752   1,376.0   1,376.0     1,312     1,440         90.5  assign_req_to_token_pool                                                                            
+                    266  266          0          2            2,657   1,328.5   1,328.5     1,152     1,505        249.6  get_last_loc_kernel                                                                                 
+                    266  266          0          1            2,112   2,112.0   2,112.0     2,112     2,112          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    266  266          0          1            1,248   1,248.0   1,248.0     1,248     1,248          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+                    267  267          0         19          287,978  15,156.7  15,137.0    14,784    15,872        289.5  create_flashinfer_kv_indices_triton                                                                 
+                    267  267          0         40          150,758   3,768.9   2,448.0     1,312     7,424      1,922.2  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    267  267          0         40           94,307   2,357.7   2,416.5     1,312     2,688        272.4  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    267  267          0         10           42,721   4,272.1   4,704.0     1,344     5,600      1,500.9  alloc_extend_kernel                                                                                 
+                    267  267          0         35           38,630   1,103.7   1,120.0       768     1,697        218.3  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    267  267          0         22           32,896   1,495.3   1,440.0     1,376     2,144        173.9  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    267  267          0         19           28,256   1,487.2   1,472.0     1,440     1,568         39.0  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    267  267          0         19           22,401   1,179.0   1,184.0     1,152     1,280         35.8  _fused_replay_state_indices_kernel                                                                  
+                    267  267          0         19           19,840   1,044.2   1,024.0       992     1,152         37.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    267  267          0         19           16,001     842.2     832.0       832       864         15.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    267  267          0         19           13,600     715.8     704.0       672       896         54.6  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    267  267          0          8           12,480   1,560.0   1,536.0     1,472     1,728         83.4  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    267  267          0          8           10,368   1,296.0   1,152.0     1,120     2,400        446.4  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    267  267          0          2            2,912   1,456.0   1,456.0     1,344     1,568        158.4  assign_req_to_token_pool                                                                            
+                    267  267          0          2            2,784   1,392.0   1,392.0     1,280     1,504        158.4  get_last_loc_kernel                                                                                 
+                    267  267          0          1            2,272   2,272.0   2,272.0     2,272     2,272          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    267  267          0          1            1,216   1,216.0   1,216.0     1,216     1,216          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+                    268  268          0         19          287,625  15,138.2  15,072.0    14,816    15,681        238.3  create_flashinfer_kv_indices_triton                                                                 
+                    268  268          0         40          161,350   4,033.8   2,464.0     1,312     7,840      2,053.7  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    268  268          0         40           94,273   2,356.8   2,496.0     1,312     2,784        304.2  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    268  268          0         10           43,586   4,358.6   4,800.0     1,344     5,569      1,523.1  alloc_extend_kernel                                                                                 
+                    268  268          0         35           39,009   1,114.5   1,120.0       768     1,568        207.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    268  268          0         22           33,760   1,534.5   1,472.0     1,408     2,112        160.3  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+                    268  268          0         19           27,620   1,453.7   1,440.0     1,376     1,536         35.9  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+                    268  268          0         19           22,752   1,197.5   1,184.0     1,152     1,280         35.9  _fused_replay_state_indices_kernel                                                                  
+                    268  268          0         19           20,096   1,057.7   1,056.0     1,024     1,088         19.9  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+                    268  268          0         19           16,288     857.3     864.0       832       864         13.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+                    268  268          0         19           13,888     730.9     704.0       704       928         54.7  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+                    268  268          0          8           12,801   1,600.1   1,584.0     1,504     1,697         66.5  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+                    268  268          0          8           10,304   1,288.0   1,152.0     1,120     2,272        397.8  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+                    268  268          0          2            3,008   1,504.0   1,504.0     1,408     1,600        135.8  assign_req_to_token_pool                                                                            
+                    268  268          0          2            2,721   1,360.5   1,360.5     1,152     1,569        294.9  get_last_loc_kernel                                                                                 
+                    268  268          0          1            2,048   2,048.0   2,048.0     2,048     2,048          0.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+                    268  268          0          1            1,216   1,216.0   1,216.0     1,216     1,216          0.0  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+ :dflash_a…  Push…  261  261         20         20          160,771   8,038.6   8,064.0     7,936     8,160         60.2  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  261  261         20         20           48,066   2,403.3   2,400.0     2,336     2,593         54.0  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_a…  Push…  262  262         19         19          151,973   7,998.6   7,968.0     7,904     8,225         79.6  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  262  262         19         19           44,322   2,332.7   2,304.0     2,272     2,464         60.2  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_a…  Push…  263  263         19         19          152,969   8,051.0   8,033.0     7,937     8,193         70.1  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  263  263         19         19           46,915   2,469.2   2,432.0     2,400     2,592         65.2  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_a…  Push…  264  264         19         19          152,162   8,008.5   8,000.0     7,904     8,160         81.1  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  264  264         19         19           47,041   2,475.8   2,432.0     2,400     2,656         76.3  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_a…  Push…  265  265         19         19          153,381   8,072.7   8,064.0     7,968     8,256         69.9  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  265  265         19         19           46,624   2,453.9   2,432.0     2,400     2,592         47.8  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_a…  Push…  266  266         19         19          154,884   8,151.8   8,160.0     8,032     8,384         97.7  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  266  266         19         19           44,193   2,325.9   2,304.0     2,240     2,528         68.4  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_a…  Push…  267  267         19         19          153,575   8,082.9   8,065.0     7,905     8,289        103.5  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  267  267         19         19           45,827   2,411.9   2,400.0     2,368     2,496         38.7  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_a…  Push…  268  268         19         19          151,042   7,949.6   7,968.0     7,872     8,032         45.7  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+ :dflash_a…  Push…  268  268         19         19           47,106   2,479.3   2,464.0     2,432     2,624         50.5  _dflash_accept_bonus_contig_kernel                                                                  
+ :dflash_d…  Push…  261  261         20        260      140,794,412  541,517…  271,096…   216,455  52,394,…  3,289,463.0  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  261  261         20        120       36,515,931  304,299…  305,162…   284,937   309,354      3,720.2  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  261  261         20         40        3,633,872  90,846.8  89,763.0    71,554   138,884     13,082.0  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  261  261         20        240        2,261,312   9,422.1  10,720.0     5,888    16,768      3,072.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  261  261         20         20        1,259,143  62,957.2  62,930.0    61,954    65,570        737.9  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  261  261         20        120          854,677   7,122.3   7,024.0     6,561     9,760        492.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  261  261         20        260          574,224   2,208.6   2,240.5     1,568     2,336        182.2  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  261  261         20        240          562,578   2,344.1   2,336.0     2,240     2,560         54.8  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  261  261         20        120          388,234   3,235.3   3,184.0     3,008     4,033        192.6  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  261  261         20        240          377,353   1,572.3   1,536.5     1,408     1,888        123.8  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  261  261         20        120          363,851   3,032.1   3,040.0     2,784     3,136         43.0  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  261  261         20         40          352,554   8,813.9   5,568.0     2,912    15,745      6,067.4  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  261  261         20         20          190,054   9,502.7   9,504.0     9,409     9,632         52.3  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  261  261         20        120          148,551   1,237.9   1,216.0     1,152     1,536         76.5  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  261  261         20        120          133,187   1,109.9   1,088.0     1,024     1,888        125.1  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  261  261         20         20           76,066   3,803.3   3,744.0     3,680     4,001        124.3  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  261  261         20         60           72,642   1,210.7   1,040.5       992     1,697        262.3  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  261  261         20         60           56,805     946.8     960.0       736     1,312        153.1  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  261  261         20         20           42,402   2,120.1   2,112.0     2,080     2,208         40.0  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  261  261         20         60           35,776     596.3     528.0       512       800        110.1  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  261  261         20         20           31,970   1,598.5   1,568.5     1,536     1,760         60.9  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  261  261         20         20           27,488   1,374.4   1,376.0     1,184     1,536        105.6  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  261  261         20         20           27,201   1,360.0   1,344.0     1,184     1,568         84.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  261  261         20         20           26,913   1,345.7   1,344.0     1,312     1,440         33.6  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  261  261         20         20           24,416   1,220.8   1,344.0       896     1,536        249.9  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  261  261         20         20           21,474   1,073.7   1,056.0     1,024     1,153         28.6  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  261  261         20         20           20,865   1,043.3   1,056.0       992     1,120         26.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  261  261         20         20           20,640   1,032.0   1,024.0       992     1,184         41.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  261  261         20         20           19,745     987.3     992.0       960     1,024         15.7  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  261  261         20         20           13,920     696.0     704.0       672       736         20.4  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_d…  Push…  262  262         19        247       87,490,381  354,212…  273,575…   226,502  12,461,…    819,109.8  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  262  262         19        114       34,132,683  299,409…  299,048…   295,272   305,705      1,956.1  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  262  262         19         38        3,422,078  90,054.7  88,434.5    73,122   142,692     13,016.6  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  262  262         19        228        2,105,623   9,235.2  10,272.5     5,824    16,993      3,015.5  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  262  262         19         19        1,193,856  62,834.5  62,754.0    62,338    64,002        402.7  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  262  262         19        114          818,099   7,176.3   7,120.0     6,592     8,960        432.7  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  262  262         19        247          538,122   2,178.6   2,208.0     1,568     2,305        179.8  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  262  262         19        228          523,692   2,296.9   2,273.0     2,208     2,624         56.2  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  262  262         19        114          362,118   3,176.5   3,104.0     3,008     3,809        177.5  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  262  262         19        228          351,372   1,541.1   1,536.0     1,408     1,856        127.0  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  262  262         19        114          340,589   2,987.6   2,992.5     2,784     3,072         44.2  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  262  262         19         38          337,862   8,891.1   5,584.0     2,880    15,617      6,089.8  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  262  262         19         19          179,494   9,447.1   9,441.0     9,344     9,600         76.6  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  262  262         19        114          141,410   1,240.4   1,216.0     1,120     1,504         81.8  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  262  262         19        114          125,956   1,104.9   1,056.0     1,024     1,792        110.0  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  262  262         19         19           70,566   3,714.0   3,712.0     3,648     3,840         44.6  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  262  262         19         57           68,193   1,196.4   1,024.0       992     1,632        254.8  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  262  262         19         57           53,794     943.8     960.0       736     1,216        146.1  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  262  262         19         38           42,432   1,116.6   1,120.0     1,024     1,216         63.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  262  262         19         19           40,193   2,115.4   2,080.0     2,016     2,304         68.2  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  262  262         19         57           34,528     605.8     544.0       512       960        125.7  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  262  262         19         19           30,688   1,615.2   1,600.0     1,536     1,792         70.2  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  262  262         20         20           27,744   1,387.2   1,392.0     1,152     1,536         72.9  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  262  262         19         19           27,587   1,451.9   1,440.0     1,216     1,632         90.1  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  262  262         19         19           24,961   1,313.7   1,312.0     1,280     1,376         29.2  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  262  262         19         19           20,064   1,056.0   1,024.0       992     1,184         51.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  262  262         19         19           19,746   1,039.3   1,024.0       992     1,216         50.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  262  262         20         20           19,265     963.3     928.0       896     1,344        131.7  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  262  262         19         19           18,817     990.4     992.0       929     1,120         44.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  262  262         19         19           13,408     705.7     704.0       672       832         39.2  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_d…  Push…  263  263         19        247       85,594,391  346,536…  270,728…   226,983  11,108,…    738,478.8  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  263  263         19        114       34,611,162  303,606…  304,121…   296,809   309,034      2,610.9  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  263  263         19         38        3,456,843  90,969.6  90,706.5    75,522   128,835     11,687.4  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  263  263         19        228        2,112,961   9,267.4  10,464.0     5,856    17,249      3,029.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  263  263         19         19        1,194,758  62,882.0  62,754.0    62,466    63,874        362.8  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  263  263         19        114          813,916   7,139.6   7,104.0     6,592     8,992        401.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  263  263         19        247          541,676   2,193.0   2,208.0     1,568     2,337        183.4  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  263  263         19        228          529,992   2,324.5   2,304.0     2,176     2,624         64.6  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  263  263         19        114          361,741   3,173.2   3,136.0     3,008     3,841        154.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  263  263         19        228          355,566   1,559.5   1,536.5     1,408     1,952        130.2  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  263  263         19        114          345,319   3,029.1   3,040.0     2,912     3,104         31.3  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  263  263         19         38          339,688   8,939.2   5,616.0     2,944    16,064      6,111.3  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  263  263         19         19          181,223   9,538.1   9,536.0     9,472     9,664         44.6  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  263  263         19        114          142,565   1,250.6   1,216.0     1,152     1,728         92.2  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  263  263         19        114          127,458   1,118.1   1,088.0     1,024     1,888        132.4  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  263  263         19         19           71,681   3,772.7   3,776.0     3,712     3,936         47.6  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  263  263         19         57           68,867   1,208.2   1,056.0       960     1,664        253.3  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  263  263         19         57           54,019     947.7     960.0       768     1,344        143.4  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  263  263         19         38           43,424   1,142.7   1,152.0     1,024     1,376         80.1  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  263  263         19         19           40,322   2,122.2   2,112.0     2,048     2,240         47.7  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  263  263         19         57           34,370     603.0     513.0       512       928        124.6  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  263  263         19         19           30,720   1,616.8   1,600.0     1,536     1,760         50.4  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  263  263         20         20           28,673   1,433.7   1,424.0     1,152     1,600         95.0  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  263  263         19         19           26,752   1,408.0   1,408.0     1,216     1,568         79.1  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  263  263         19         19           24,896   1,310.3   1,312.0     1,248     1,472         51.7  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  263  263         19         19           20,449   1,076.3   1,056.0       992     1,216         71.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  263  263         19         19           20,065   1,056.1   1,056.0     1,024     1,216         41.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  263  263         19         19           18,945     997.1     992.0       960     1,120         37.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  263  263         20         20           18,656     932.8     896.0       896     1,408        112.9  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  263  263         19         19           13,443     707.5     704.0       672       768         33.5  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_d…  Push…  264  264         19        247       84,688,100  342,866…  271,015…   244,231  10,379,…    695,343.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  264  264         19        114       34,887,481  306,030…  306,120…   302,536   309,608      1,425.5  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  264  264         19         38        3,210,455  84,485.7  83,506.0    74,562   101,250      6,090.0  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  264  264         19        228        2,122,590   9,309.6  10,896.0     5,856    16,545      3,037.6  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  264  264         19         19        1,197,184  63,009.7  62,946.0    62,242    64,514        489.4  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  264  264         19        114          809,586   7,101.6   7,040.0     6,592     9,536        407.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  264  264         19        247          542,195   2,195.1   2,240.0     1,600     2,336        174.6  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  264  264         19        228          531,084   2,329.3   2,304.0     2,240     2,497         46.0  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  264  264         19        114          363,658   3,190.0   3,168.0     3,008     3,584        138.1  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  264  264         19        228          360,870   1,582.8   1,568.0     1,440     1,984        141.5  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  264  264         19        114          344,590   3,022.7   3,040.0     2,912     3,072         32.8  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  264  264         19         38          342,891   9,023.4   5,856.0     2,912    15,616      6,129.0  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  264  264         19         19          182,470   9,603.7   9,600.0     9,536     9,696         43.7  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  264  264         19        114          140,997   1,236.8   1,216.0     1,152     1,536         79.2  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  264  264         19        114          129,666   1,137.4   1,120.0     1,056     1,888        119.9  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  264  264         19         19           71,938   3,786.2   3,776.0     3,776     3,809         15.4  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  264  264         19         57           70,593   1,238.5   1,056.0     1,024     1,696        259.3  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  264  264         19         57           55,554     974.6     992.0       768     1,216        148.5  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  264  264         19         38           44,704   1,176.4   1,152.0     1,056     1,472        113.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  264  264         19         19           40,258   2,118.8   2,081.0     2,048     2,272         67.1  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  264  264         19         57           34,947     613.1     544.0       512       960        118.2  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  264  264         19         19           31,936   1,680.8   1,696.0     1,600     1,792         57.8  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  264  264         20         20           28,673   1,433.7   1,424.0     1,184     1,600         84.7  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  264  264         19         19           27,808   1,463.6   1,472.0     1,248     1,632         83.2  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  264  264         19         19           24,768   1,303.6   1,312.0     1,248     1,376         33.5  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  264  264         19         19           20,449   1,076.3   1,056.0     1,024     1,312         61.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  264  264         19         19           19,680   1,035.8   1,024.0     1,024     1,056         15.9  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  264  264         19         19           19,648   1,034.1   1,024.0       992     1,216         48.9  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  264  264         20         20           18,465     923.3     896.0       896     1,312         92.4  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  264  264         19         19           14,080     741.1     736.0       704       800         26.7  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_d…  Push…  265  265         19        247       84,729,271  343,033…  271,016…   262,247  10,613,…    707,602.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  265  265         19        114       34,980,506  306,846…  306,904…   303,368   310,281      1,570.2  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  265  265         19         38        2,723,407  71,668.6  74,258.0    53,761    94,787     14,325.6  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  265  265         19        228        2,122,915   9,311.0  10,672.0     5,888    16,928      2,951.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  265  265         19         19        1,185,602  62,400.1  62,210.0    61,922    63,522        445.1  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  265  265         19        114          814,874   7,148.0   7,088.5     6,592     8,992        375.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  265  265         19        247          543,949   2,202.2   2,240.0     1,600     2,337        176.5  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  265  265         19        228          535,185   2,347.3   2,336.0     2,240     2,656         54.8  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  265  265         19        114          370,407   3,249.2   3,200.0     3,072     3,840        156.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  265  265         19        228          359,913   1,578.6   1,552.5     1,440     1,856        121.8  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  265  265         19        114          348,327   3,055.5   3,072.0     2,912     3,136         38.1  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  265  265         19         38          339,303   8,929.0   5,568.5     2,944    15,776      6,084.2  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  265  265         19         19          182,949   9,628.9   9,632.0     9,536     9,792         71.3  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  265  265         19        114          141,605   1,242.1   1,216.0     1,152     1,568         78.1  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  265  265         19        114          128,293   1,125.4   1,088.0     1,024     1,856        122.6  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  265  265         19         19           72,418   3,811.5   3,712.0     3,648     4,192        170.6  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  265  265         19         57           70,018   1,228.4   1,056.0     1,024     1,824        264.5  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  265  265         19         57           54,209     951.0     960.0       768     1,216        147.6  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  265  265         19         38           43,649   1,148.7   1,168.0     1,056     1,280         82.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  265  265         19         19           40,448   2,128.8   2,112.0     2,080     2,176         24.7  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  265  265         19         57           34,145     599.0     544.0       512       800        112.4  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  265  265         19         19           30,818   1,622.0   1,600.0     1,568     1,728         40.1  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  265  265         20         20           29,408   1,470.4   1,472.0     1,216     1,664         83.4  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  265  265         19         19           26,369   1,387.8   1,408.0     1,248     1,440         46.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  265  265         19         19           25,088   1,320.4   1,312.0     1,280     1,376         29.9  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  265  265         19         19           20,193   1,062.8   1,056.0     1,024     1,152         40.7  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  265  265         19         19           19,522   1,027.5   1,024.0       992     1,057         18.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  265  265         19         19           18,880     993.7     992.0       960     1,088         29.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  265  265         20         20           18,115     905.8     896.0       896       928         14.9  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  265  265         19         19           13,697     720.9     704.0       672       800         32.6  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_d…  Push…  266  266         19        247       83,607,964  338,493…  272,041…   262,472  9,189,2…    626,013.9  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  266  266         19        114       34,807,182  305,326…  305,418…   301,289   309,642      1,631.5  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  266  266         19         38        2,760,857  72,654.1  74,754.5    55,874    96,323     13,814.1  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  266  266         19        228        2,114,279   9,273.2  10,592.5     5,888    16,929      3,002.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  266  266         19         19        1,193,959  62,839.9  62,786.0    62,306    64,098        417.7  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  266  266         19        114          817,878   7,174.4   7,008.5     6,560     8,640        440.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  266  266         19        247          547,095   2,215.0   2,241.0     1,568     2,336        188.4  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  266  266         19        228          532,182   2,334.1   2,336.0     2,208     2,560         58.1  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  266  266         19        114          367,946   3,227.6   3,200.0     3,040     4,128        179.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  266  266         19        228          351,340   1,541.0   1,520.5     1,408     1,952        131.7  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  266  266         19        114          350,157   3,071.6   3,072.0     2,816     3,137         48.5  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  266  266         19         38          334,953   8,814.6   5,568.0     2,816    15,488      6,037.7  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  266  266         19         19          180,770   9,514.2   9,504.0     9,440     9,664         59.4  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  266  266         19        114          142,949   1,253.9   1,216.0     1,152     1,472         70.6  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  266  266         19        114          125,668   1,102.4   1,056.0     1,024     1,824        127.0  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  266  266         19         19           71,269   3,751.0   3,744.0     3,712     3,872         40.6  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  266  266         19         57           67,907   1,191.4   1,024.0       992     1,664        252.4  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  266  266         19         57           53,668     941.5     960.0       736     1,248        144.5  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  266  266         19         38           42,275   1,112.5   1,120.5     1,024     1,280         60.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  266  266         19         19           40,898   2,152.5   2,112.0     2,049     2,464        106.0  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  266  266         19         57           33,955     595.7     512.0       512       800        118.0  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  266  266         19         19           30,913   1,627.0   1,632.0     1,568     1,856         66.8  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  266  266         20         20           27,808   1,390.4   1,376.0     1,152     1,568         75.2  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  266  266         19         19           27,138   1,428.3   1,440.0     1,216     1,600         66.8  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  266  266         19         19           25,184   1,325.5   1,312.0     1,280     1,376         26.8  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  266  266         19         19           19,521   1,027.4   1,024.0       992     1,216         49.9  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  266  266         20         20           19,425     971.3     928.0       896     1,408        148.0  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  266  266         19         19           19,264   1,013.9   1,024.0       992     1,056         18.6  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  266  266         19         19           18,753     987.0     992.0       960     1,056         24.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  266  266         19         19           12,865     677.1     672.0       672       704         12.0  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_d…  Push…  267  267         19        247       81,429,687  329,674…  275,497…   264,617  6,266,6…    464,875.6  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  267  267         19        114       34,119,355  299,292…  299,338…   295,786   303,338      1,556.0  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  267  267         19         38        2,720,542  71,593.2  72,434.0    55,330    96,548     13,789.9  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  267  267         19        228        2,114,987   9,276.3  10,448.5     5,856    17,249      3,025.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  267  267         19         19        1,196,299  62,963.1  62,914.0    62,338    64,002        513.4  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  267  267         19        114          811,192   7,115.7   7,040.0     6,593     8,865        399.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  267  267         19        247          537,204   2,174.9   2,208.0     1,568     2,304        173.0  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  267  267         19        228          521,489   2,287.2   2,272.0     2,144     2,465         57.0  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  267  267         19        114          361,357   3,169.8   3,104.0     3,008     3,616        155.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  267  267         19        228          350,639   1,537.9   1,520.5     1,408     1,824        128.7  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  267  267         19        114          339,021   2,973.9   2,976.0     2,784     3,072         41.9  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  267  267         19         38          338,663   8,912.2   5,424.0     2,880    15,873      6,116.1  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  267  267         19         19          179,782   9,462.2   9,472.0     9,312     9,664         66.5  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  267  267         19        114          141,283   1,239.3   1,216.0     1,152     1,504         83.1  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  267  267         19        114          126,340   1,108.2   1,057.0     1,024     1,856        134.1  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  267  267         19         19           70,498   3,710.4   3,712.0     3,680     3,808         27.1  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  267  267         19         57           67,779   1,189.1   1,056.0       960     1,633        255.8  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  267  267         19         57           53,410     937.0     960.0       736     1,217        148.4  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  267  267         19         38           43,139   1,135.2   1,104.0     1,024     1,408        102.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  267  267         19         19           39,650   2,086.8   2,080.0     2,048     2,240         47.1  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  267  267         19         57           33,761     592.3     512.0       480       768        112.0  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  267  267         19         19           30,305   1,595.0   1,568.0     1,536     1,697         41.8  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  267  267         20         20           27,682   1,384.1   1,376.5     1,185     1,504         56.6  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  267  267         19         19           25,986   1,367.7   1,344.0     1,216     1,569         73.1  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  267  267         19         19           25,152   1,323.8   1,312.0     1,248     1,440         46.8  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  267  267         19         19           19,712   1,037.5   1,024.0       992     1,120         30.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  267  267         19         19           19,520   1,027.4   1,024.0       992     1,152         36.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  267  267         19         19           18,753     987.0     992.0       960     1,120         37.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  267  267         20         20           18,336     916.8     896.0       864     1,344        102.4  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  267  267         19         19           13,249     697.3     673.0       672       768         34.7  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_d…  Push…  268  268         19        247       84,181,786  340,816…  273,672…   224,839  9,994,3…    672,706.8  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_d…  Push…  268  268         19        114       34,381,182  301,589…  301,497…   298,249   305,961      1,436.0  _fwd_kernel                                                                                         
+ :dflash_d…  Push…  268  268         19         38        2,733,676  71,938.8  72,466.0    60,993    94,755      8,566.7  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_d…  Push…  268  268         19        228        2,117,720   9,288.2  10,592.5     5,856    16,705      3,051.6  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_d…  Push…  268  268         19         19        1,203,365  63,335.0  62,498.0    61,858    79,522      3,941.7  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_d…  Push…  268  268         19        114          814,526   7,145.0   7,104.0     6,593     9,505        414.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_d…  Push…  268  268         19        247          538,032   2,178.3   2,208.0     1,600     2,336        165.8  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_d…  Push…  268  268         19        228          524,687   2,301.3   2,304.0     2,208     2,496         44.8  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+ :dflash_d…  Push…  268  268         19        114          368,422   3,231.8   3,200.0     3,008     4,512        215.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_d…  Push…  268  268         19        228          356,137   1,562.0   1,552.0     1,408     1,824        131.2  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_d…  Push…  268  268         19         38          340,619   8,963.7   5,472.0     2,912    15,808      6,123.3  create_flashinfer_kv_indices_triton                                                                 
+ :dflash_d…  Push…  268  268         19        114          340,425   2,986.2   2,977.0     2,816     3,040         34.1  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+ :dflash_d…  Push…  268  268         19         19          181,923   9,574.9   9,600.0     9,344     9,696         87.1  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+ :dflash_d…  Push…  268  268         19        114          139,874   1,227.0   1,184.5     1,152     1,600         82.9  _table_qk_norm_rope_kernel                                                                          
+ :dflash_d…  Push…  268  268         19        114          126,370   1,108.5   1,088.0     1,024     1,856        141.8  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+ :dflash_d…  Push…  268  268         19         19           70,594   3,715.5   3,712.0     3,680     3,776         25.8  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+ :dflash_d…  Push…  268  268         19         57           69,923   1,226.7   1,056.0       992     1,696        266.1  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ :dflash_d…  Push…  268  268         19         57           54,978     964.5     992.0       768     1,152        145.5  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  268  268         19         38           44,068   1,159.7   1,152.5     1,024     1,472        109.4  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_d…  Push…  268  268         19         19           39,235   2,065.0   2,048.0     2,016     2,208         49.4  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_d…  Push…  268  268         19         57           34,498     605.2     544.0       512       992        119.5  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ :dflash_d…  Push…  268  268         19         19           30,977   1,630.4   1,600.0     1,536     1,792         75.8  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+ :dflash_d…  Push…  268  268         20         20           29,152   1,457.6   1,472.0     1,184     1,600        103.0  _prepare_dflash_draft_block_contig_kernel                                                           
+ :dflash_d…  Push…  268  268         19         19           27,169   1,429.9   1,440.0     1,248     1,504         55.5  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_d…  Push…  268  268         19         19           24,001   1,263.2   1,280.0     1,216     1,312         28.9  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+ :dflash_d…  Push…  268  268         19         19           20,128   1,059.4   1,056.0     1,024     1,184         38.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  268  268         19         19           19,970   1,051.1   1,056.0       992     1,184         44.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+ :dflash_d…  Push…  268  268         19         19           19,712   1,037.5   1,024.0       992     1,088         22.2  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+ :dflash_d…  Push…  268  268         20         20           17,921     896.0     896.0       864       960         27.4  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_d…  Push…  268  268         19         19           13,408     705.7     704.0       672       800         31.1  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+ :dflash_m…  Push…  261  261         20         20       10,291,457  514,572…  513,632…   512,752   531,088      3,954.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  261  261         20         20          394,699  19,735.0  19,760.5    18,944    20,385        402.7  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  261  261         20        120          122,851   1,023.8     944.0       832     1,569        218.8  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  261  261         20         40          117,765   2,944.1   2,864.0     1,824     4,416      1,111.6  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  261  261         20         20          101,124   5,056.2   4,880.0     4,672     5,600        349.2  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  261  261         20         20           33,635   1,681.8   1,664.0     1,601     1,793         56.3  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_m…  Push…  262  262         19         19        9,778,859  514,676…  514,542…   513,582   516,558        854.4  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  262  262         19         19          375,917  19,785.1  19,713.0    19,233    20,353        345.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  262  262         19        114          117,284   1,028.8     944.0       864     1,632        233.4  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  262  262         19         38          110,788   2,915.5   2,784.5     1,792     4,320      1,096.1  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  262  262         19         19           89,088   4,688.8   4,576.0     4,416     5,792        331.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  262  262         19         19           32,256   1,697.7   1,696.0     1,600     2,048         98.6  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_m…  Push…  263  263         19         19        9,782,667  514,877…  514,640…   513,328   516,656        908.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  263  263         19         19          374,189  19,694.2  19,712.0    19,297    20,257        308.1  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  263  263         19        114          118,916   1,043.1     944.0       864     1,664        231.2  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  263  263         19         38          109,987   2,894.4   2,832.0     1,824     4,384      1,063.5  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  263  263         19         19           90,113   4,742.8   4,608.0     4,448     5,792        374.0  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  263  263         19         19           33,313   1,753.3   1,728.0     1,632     1,953         95.9  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_m…  Push…  264  264         19         19        9,769,961  514,208…  514,062…   512,653   516,462      1,055.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  264  264         19         19          373,515  19,658.7  19,648.0    19,297    20,512        258.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  264  264         19        114          119,586   1,049.0     960.0       864     1,632        221.4  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  264  264         19         38          111,267   2,928.1   2,848.5     1,824     4,320      1,099.9  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  264  264         19         19           91,490   4,815.3   4,736.0     4,512     5,664        314.1  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  264  264         19         19           32,226   1,696.1   1,696.0     1,632     1,792         44.0  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_m…  Push…  265  265         19         19        9,762,651  513,823…  514,031…   512,207   516,239      1,089.1  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  265  265         19         19          377,355  19,860.8  19,840.0    19,201    20,289        298.4  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  265  265         19        114          118,626   1,040.6     960.0       864     1,600        223.4  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  265  265         19         38          113,412   2,984.5   2,864.5     1,824     4,353      1,138.9  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  265  265         19         19           95,648   5,034.1   4,832.0     4,544     5,952        449.4  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  265  265         19         19           32,481   1,709.5   1,696.0     1,664     1,856         55.8  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_m…  Push…  266  266         19         19        9,785,343  515,018…  514,992…   514,033   516,401        686.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  266  266         19         19          374,956  19,734.5  19,744.0    19,137    20,064        226.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  266  266         19        114          118,821   1,042.3     960.5       896     1,632        214.1  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  266  266         19         38          111,233   2,927.2   2,848.0     1,856     4,352      1,077.4  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  266  266         19         19           92,674   4,877.6   4,704.0     4,448     5,665        405.9  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  266  266         19         19           32,417   1,706.2   1,696.0     1,600     1,888         73.2  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_m…  Push…  267  267         19         19        9,771,176  514,272…  514,545…   513,553   515,058        574.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  267  267         19         19          375,661  19,771.6  19,745.0    19,329    20,417        252.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  267  267         19        114          117,027   1,026.6     944.0       864     1,568        225.9  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  267  267         19         38          109,571   2,883.4   2,800.0     1,824     4,448      1,067.4  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  267  267         19         19           92,196   4,852.4   4,608.0     4,352     5,632        477.4  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  267  267         19         19           32,256   1,697.7   1,664.0     1,632     1,792         47.1  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_m…  Push…  268  268         19         19        9,778,586  514,662…  514,351…   513,422   517,007      1,007.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+ :dflash_m…  Push…  268  268         19         19          375,787  19,778.3  19,776.0    19,296    21,505        492.4  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+ :dflash_m…  Push…  268  268         19        114          116,804   1,024.6     928.0       832     1,600        231.3  set_kv_buffer_prefix_valid_tiled                                                                    
+ :dflash_m…  Push…  268  268         19         38          111,041   2,922.1   2,864.0     1,792     4,352      1,117.3  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_m…  Push…  268  268         19         19           94,021   4,948.5   4,704.0     4,545     5,761        423.7  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_m…  Push…  268  268         19         19           33,248   1,749.9   1,696.0     1,632     1,920         88.0  _fused_norm_rope_kernel_stacked                                                                     
+ :dflash_r…  Push…  261  261         20         20          280,391  14,019.5  13,856.0    13,441    17,088        753.1  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  261  261         20         20           80,546   4,027.3   3,936.0     3,680     5,089        331.4  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  261  261         20         20           27,905   1,395.3   1,376.0     1,344     1,569         66.9  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  261  261         20         20           22,784   1,139.2   1,120.0     1,088     1,344         66.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  261  261         20         20           21,696   1,084.8   1,056.0     1,056     1,216         43.9  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_r…  Push…  262  262         19         19          270,248  14,223.6  14,208.0    13,536    14,784        339.4  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  262  262         19         19           73,189   3,852.1   3,808.0     3,585     4,256        192.9  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  262  262         19         19           27,072   1,424.8   1,408.0     1,408     1,504         30.9  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  262  262         19         19           20,929   1,101.5   1,088.0     1,056     1,184         26.8  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  262  262         19         19           20,288   1,067.8   1,056.0     1,024     1,120         24.3  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_r…  Push…  263  263         19         19          266,729  14,038.4  14,112.0    13,217    14,496        356.7  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  263  263         19         19           73,475   3,867.1   3,808.0     3,520     4,480        263.2  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  263  263         19         19           27,232   1,433.3   1,440.0     1,408     1,536         31.2  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  263  263         19         19           21,056   1,108.2   1,088.0     1,056     1,248         41.6  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  263  263         19         19           20,224   1,064.4   1,056.0     1,024     1,184         36.7  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_r…  Push…  264  264         19         19          261,702  13,773.8  13,792.0    13,473    14,240        226.9  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  264  264         19         19           75,620   3,980.0   4,000.0     3,616     4,352        224.6  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  264  264         19         19           26,593   1,399.6   1,376.0     1,344     1,536         48.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  264  264         19         19           21,313   1,121.7   1,120.0     1,088     1,153         22.6  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  264  264         19         19           20,385   1,072.9   1,056.0     1,056     1,184         30.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_r…  Push…  265  265         19         19          273,546  14,397.2  14,337.0    13,985    14,912        226.5  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  265  265         19         19           74,497   3,920.9   3,840.0     3,648     4,640        262.0  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  265  265         19         19           27,041   1,423.2   1,408.0     1,344     1,536         43.2  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  265  265         19         19           21,281   1,120.1   1,120.0     1,056     1,216         44.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  265  265         19         19           20,609   1,084.7   1,056.0     1,025     1,216         49.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_r…  Push…  266  266         19         19          271,561  14,292.7  14,337.0    13,697    15,201        375.3  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  266  266         19         19           76,739   4,038.9   4,064.0     3,712     4,385        222.2  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  266  266         19         19           26,784   1,409.7   1,408.0     1,376     1,472         25.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  266  266         19         19           21,185   1,115.0   1,120.0     1,056     1,184         35.8  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  266  266         19         19           20,640   1,086.3   1,056.0     1,024     1,248         63.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_r…  Push…  267  267         19         19          265,960  13,997.9  14,144.0    13,281    14,560        398.9  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  267  267         19         19           75,073   3,951.2   3,936.0     3,616     4,352        227.1  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  267  267         19         19           27,424   1,443.4   1,408.0     1,376     1,664         79.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  267  267         19         19           21,250   1,118.4   1,088.0     1,056     1,280         64.4  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  267  267         19         19           19,905   1,047.6   1,024.0     1,024     1,152         41.1  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_r…  Push…  268  268         19         19          274,248  14,434.1  14,400.0    14,016    14,817        231.0  _fused_mamba_state_scatter_with_mask_kernel                                                         
+ :dflash_r…  Push…  268  268         19         19           74,306   3,910.8   3,936.0     3,616     4,320        217.6  _fused_conv_window_scatter_multi_kernel                                                             
+ :dflash_r…  Push…  268  268         19         19           28,001   1,473.7   1,472.0     1,440     1,568         29.2  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  268  268         19         19           21,121   1,111.6   1,088.0     1,088     1,216         39.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_r…  Push…  268  268         19         19           19,681   1,035.8   1,024.0     1,024     1,056         15.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+ :dflash_t…  Push…  261  261         20        480    1,877,141,617  3,910,7…  3,910,0…  3,876,2…  3,942,7…      9,512.4  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  261  261         20      3,740    1,121,980,937  299,994…  291,977…   215,750  1,913,4…     56,045.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  261  261         20      3,680      130,290,708  35,405.1  32,705.0    13,281    70,562     11,552.1  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  261  261         20      1,840      110,696,805  60,161.3  60,098.0    57,634    70,243        879.6  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  261  261         20      4,200       99,481,220  23,686.0   6,433.0     4,032    50,530     20,181.1  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  261  261         20         20       67,938,338  3,396,9…  1,641,7…  1,376,3…  9,960,0…  3,012,197.1  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  261  261         20      1,380       55,544,329  40,249.5  40,321.0    39,201    41,025        309.3  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  261  261         20      3,240       47,146,943  14,551.5  18,369.0     5,281    23,393      7,315.5  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  261  261         20      3,840       24,533,309   6,388.9   6,272.0     4,448     7,937        635.1  _score_kernel                                                                                       
+ :dflash_t…  Push…  261  261         20      1,840       15,361,090   8,348.4   8,288.0     5,600    13,344      1,023.0  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  261  261         20        480       13,164,976  27,427.0  27,409.0    25,889    29,377        586.2  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  261  261         20      2,320        9,049,727   3,900.7   4,160.0     2,432     4,832        649.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  261  261         20      3,260        8,615,555   2,642.8   2,721.0     1,760     5,345        514.9  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  261  261         20      3,840        7,985,648   2,079.6   2,112.0       928     3,136        515.4  _combine_kernel                                                                                     
+ :dflash_t…  Push…  261  261         20      3,740        7,797,106   2,084.8   2,080.0     1,920     2,432         68.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  261  261         20      1,380        7,358,941   5,332.6   5,377.0     4,864     6,560        226.9  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  261  261         20      1,840        7,062,418   3,838.3   3,839.0     3,648     4,160         72.9  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  261  261         20      1,840        6,453,367   3,507.3   3,488.0     3,328     3,872         84.5  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  261  261         20      1,840        4,924,853   2,676.6   2,656.0     2,368     3,233        108.2  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  261  261         20      1,840        4,383,842   2,382.5   2,368.0     2,272     2,784         52.9  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  261  261         20      1,840        4,235,322   2,301.8   2,304.0     2,208     2,688         55.3  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  261  261         20      1,840        3,825,936   2,079.3   2,080.0     1,920     2,816         66.9  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  261  261         20      1,380        3,660,171   2,652.3   2,624.0     2,368     3,136         88.2  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  261  261         20      1,840        3,508,265   1,906.7   1,888.0     1,824     2,208         38.8  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  261  261         20      1,840        3,478,154   1,890.3   1,888.0     1,792     2,143         39.0  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  261  261         20      1,840        3,360,673   1,826.5   1,824.0     1,760     2,048         35.8  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  261  261         20      1,840        3,280,586   1,782.9   1,761.0     1,696     2,048         47.5  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  261  261         20      1,380        3,012,092   2,182.7   2,176.0     2,080     2,496         56.4  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  261  261         20      1,560        3,001,427   1,924.0   1,888.0     1,664     5,600        416.9  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  261  261         20        480        2,942,905   6,131.1   6,113.0     5,920     6,528        102.0  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  261  261         20      1,840        2,673,305   1,452.9   1,440.0     1,248     2,049        117.1  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  261  261         20      1,840        2,411,066   1,310.4   1,280.0     1,216     1,728         69.6  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  261  261         20        500        2,256,293   4,512.6   3,488.0     3,168    26,945      4,491.4  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  261  261         20      1,840        2,130,023   1,157.6   1,152.0     1,088     1,568         41.7  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  261  261         20      1,720        1,673,775     973.1     960.0       800     1,184         36.8  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  261  261         20        960        1,588,015   1,654.2   1,648.5     1,504     1,952         98.3  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  261  261         20        480        1,467,601   3,057.5   3,040.0     2,400     4,160        299.9  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  261  261         20         20        1,294,634  64,731.7  64,850.0    63,714    65,890        574.4  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  261  261         20        500          996,126   1,992.3   1,952.0     1,888     2,560        115.8  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  261  261         20        480          689,678   1,436.8   1,440.0     1,376     1,696         37.3  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  261  261         20        480          627,415   1,307.1   1,248.0     1,120     3,360        262.0  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  261  261         20         20          138,632   6,931.6   6,944.0     6,848     7,008         48.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  261  261         20         20           70,725   3,536.3   3,521.0     3,488     3,584         35.2  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  261  261         20         20           48,225   2,411.3   2,368.0     2,272     2,720        110.5  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  261  261         20         20           27,168   1,358.4   1,360.0     1,248     1,504         62.7  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  261  261         20         20           18,048     902.4     896.0       864     1,120         67.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  261  261         20         20           15,712     785.6     800.0       736       832         22.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ :dflash_t…  Push…  262  262         19        456    1,761,832,629  3,863,6…  3,862,3…  3,844,3…  3,893,2…      8,710.4  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  262  262         19      3,553    1,085,093,227  305,402…  292,360…   219,398  1,901,7…     59,502.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  262  262         19      3,496      123,387,268  35,293.8  32,433.0    13,504    71,361     11,598.7  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  262  262         19      1,748      105,055,331  60,100.3  60,033.0    57,889    70,146        989.6  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  262  262         19      3,990       94,454,976  23,672.9   6,400.0     4,000    50,401     20,211.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  262  262         19         19       62,325,500  3,280,2…  1,518,2…  1,375,4…  9,964,2…  3,055,437.0  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  262  262         19      1,311       51,982,981  39,651.4  39,585.0    38,850    40,673        377.3  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  262  262         19      3,078       44,940,929  14,600.7  18,816.5     5,184    23,873      7,352.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  262  262         19      3,648       23,057,274   6,320.5   6,209.0     4,416     7,872        616.3  _score_kernel                                                                                       
+ :dflash_t…  Push…  262  262         19      1,748       14,348,428   8,208.5   8,160.0     5,409    13,056      1,015.9  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  262  262         19        456       12,528,181  27,474.1  27,489.0    26,112    28,929        536.7  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  262  262         19      2,204        8,513,102   3,862.6   4,128.0     2,368     4,673        653.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  262  262         19      3,097        8,090,013   2,612.2   2,720.0     1,760     4,896        506.7  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  262  262         19      3,648        7,529,017   2,063.9   2,080.0       896     3,328        516.3  _combine_kernel                                                                                     
+ :dflash_t…  Push…  262  262         19      3,553        7,304,415   2,055.8   2,048.0     1,888     2,432         67.7  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  262  262         19      1,311        6,949,929   5,301.2   5,344.0     4,704     6,112        208.9  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  262  262         19      1,748        6,704,508   3,835.5   3,839.0     3,616     4,224         82.1  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  262  262         19      1,748        6,094,076   3,486.3   3,457.0     3,297     3,872         75.8  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  262  262         19      1,748        4,609,515   2,637.0   2,624.0     2,304     3,040        107.2  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  262  262         19      1,748        4,079,623   2,333.9   2,336.0     2,208     2,656         47.4  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  262  262         19      1,748        3,988,728   2,281.9   2,272.0     2,208     2,656         56.5  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  262  262         19      1,748        3,593,559   2,055.8   2,048.0     1,920     2,400         66.1  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  262  262         19      1,311        3,441,056   2,624.8   2,592.0     2,336     3,169         89.8  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  262  262         19      1,748        3,302,263   1,889.2   1,888.0     1,792     2,144         38.8  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  262  262         19      1,748        3,260,538   1,865.3   1,856.0     1,791     2,176         43.2  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  262  262         19      1,748        3,165,621   1,811.0   1,792.0     1,728     2,048         34.7  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  262  262         19      1,748        3,068,546   1,755.5   1,760.0     1,664     2,048         50.4  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  262  262         19      1,311        2,835,913   2,163.2   2,144.0     2,080     2,497         61.9  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  262  262         19      1,482        2,816,436   1,900.4   1,856.0     1,632     5,536        413.2  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  262  262         19        456        2,769,108   6,072.6   6,048.0     5,824     6,433        112.4  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  262  262         19      1,748        2,489,549   1,424.2   1,408.0     1,280     2,017        104.9  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  262  262         19      1,748        2,258,841   1,292.2   1,280.0     1,184     1,631         64.8  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  262  262         19        475        2,102,356   4,426.0   3,392.0     3,104    26,880      4,547.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  262  262         19      1,748        1,998,709   1,143.4   1,120.0     1,088     1,408         44.1  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  262  262         19      1,634        1,575,559     964.2     960.0       768     1,184         35.8  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  262  262         19        912        1,492,717   1,636.8   1,632.0     1,472     1,984        101.5  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  262  262         19        456        1,351,408   2,963.6   2,944.0     2,048     4,129        294.4  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  262  262         19         19        1,233,669  64,929.9  65,058.0    64,065    66,434        615.9  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  262  262         19        475          948,284   1,996.4   1,952.0     1,888     2,496        114.3  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  262  262         19        456          633,973   1,390.3   1,376.0     1,344     1,568         28.4  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  262  262         19        456          586,475   1,286.1   1,216.0     1,120     3,584        281.5  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  262  262         19         19          129,729   6,827.8   6,848.0     6,688     6,944         67.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  262  262         19         19           65,987   3,473.0   3,488.0     3,360     3,521         37.5  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  262  262         19         19           48,704   2,563.4   2,560.0     2,304     2,880        145.0  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  262  262         19         19           25,152   1,323.8   1,312.0       896     1,504        124.0  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  262  262         19         19           17,057     897.7     896.0       864       928         12.9  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  262  262         19         19           14,882     783.3     768.0       736       992         60.7  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ :dflash_t…  Push…  263  263         19        456    1,781,113,597  3,905,9…  3,904,8…  3,884,1…  3,934,6…     10,094.5  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  263  263         19      3,553    1,063,430,509  299,305…  291,785…   219,175  1,886,6…     53,338.7  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  263  263         19      3,496      123,663,167  35,372.8  32,641.0    13,345    71,650     11,595.9  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  263  263         19      1,748      105,312,942  60,247.7  60,162.0    57,698    72,610        970.3  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  263  263         19      3,990       94,522,438  23,689.8   6,432.0     4,032    49,666     20,217.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  263  263         19         19       62,388,528  3,283,6…  1,512,0…  1,374,7…  9,959,7…  3,057,633.5  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  263  263         19      1,311       52,699,894  40,198.2  40,225.0    39,234    41,089        315.7  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  263  263         19      3,078       45,024,429  14,627.8  19,024.5     5,344    23,617      7,343.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  263  263         19      3,648       23,093,951   6,330.6   6,208.0     4,448     7,969        604.1  _score_kernel                                                                                       
+ :dflash_t…  Push…  263  263         19      1,748       14,509,556   8,300.7   8,257.0     5,504    12,960      1,029.6  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  263  263         19        456       12,514,082  27,443.2  27,457.0    25,857    29,121        555.4  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  263  263         19      2,204        8,550,931   3,879.7   4,160.0     2,400     4,736        658.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  263  263         19      3,097        8,178,690   2,640.8   2,720.0     1,760     5,440        481.0  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  263  263         19      3,648        7,666,351   2,101.5   2,112.0       928     3,233        530.1  _combine_kernel                                                                                     
+ :dflash_t…  Push…  263  263         19      3,553        7,370,843   2,074.5   2,049.0     1,920     2,432         66.8  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  263  263         19      1,311        6,929,937   5,286.0   5,280.0     4,768     5,921        210.9  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  263  263         19      1,748        6,720,417   3,844.6   3,839.0     3,647     4,256         81.7  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  263  263         19      1,748        6,131,649   3,507.8   3,488.0     3,296     3,936         80.8  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  263  263         19      1,748        4,653,544   2,662.2   2,656.0     2,336     3,136        110.0  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  263  263         19      1,748        4,149,115   2,373.6   2,368.0     2,272     2,721         48.9  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  263  263         19      1,748        4,006,882   2,292.3   2,272.0     2,208     2,624         54.5  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  263  263         19      1,748        3,594,540   2,056.4   2,048.0     1,920     2,560         64.3  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  263  263         19      1,311        3,465,504   2,643.4   2,624.0     2,336     3,424         93.9  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  263  263         19      1,748        3,328,913   1,904.4   1,888.0     1,824     2,176         37.8  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  263  263         19      1,748        3,311,122   1,894.2   1,888.0     1,823     2,240         44.0  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  263  263         19      1,748        3,172,346   1,814.8   1,824.0     1,760     2,049         32.1  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  263  263         19      1,748        3,100,838   1,773.9   1,760.0     1,664     2,080         51.2  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  263  263         19      1,311        2,853,506   2,176.6   2,176.0     2,080     2,433         52.6  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  263  263         19      1,482        2,824,942   1,906.2   1,856.0     1,632     5,664        418.1  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  263  263         19        456        2,791,101   6,120.8   6,112.0     5,856     6,464        103.4  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  263  263         19      1,748        2,514,000   1,438.2   1,408.0     1,280     1,888        103.6  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  263  263         19      1,748        2,270,238   1,298.8   1,280.0     1,184     1,632         64.5  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  263  263         19        475        2,103,626   4,428.7   3,424.0     3,168    27,073      4,551.4  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  263  263         19      1,748        2,017,595   1,154.2   1,152.0     1,056     1,441         42.9  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  263  263         19      1,634        1,592,301     974.5     960.0       800     1,376         38.4  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  263  263         19        912        1,511,405   1,657.2   1,648.5     1,536     1,984         97.0  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  263  263         19        456        1,345,866   2,951.5   2,912.5     2,304     3,904        280.4  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  263  263         19         19        1,230,470  64,761.6  64,674.0    64,066    65,922        573.5  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  263  263         19        475          957,529   2,015.9   1,984.0     1,888     2,496        117.1  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  263  263         19        456          646,520   1,417.8   1,408.0     1,344     1,728         44.5  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  263  263         19        456          591,410   1,297.0   1,217.0     1,120     3,296        278.8  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  263  263         19         19          131,010   6,895.3   6,912.0     6,784     7,008         52.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  263  263         19         19           66,369   3,493.1   3,488.0     3,392     3,552         34.2  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  263  263         19         19           46,720   2,458.9   2,432.0     2,336     2,784        100.2  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  263  263         19         19           22,305   1,173.9   1,249.0       832     1,632        263.7  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  263  263         19         19           17,251     907.9     896.0       864       960         21.9  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  263  263         19         19           15,264     803.4     768.0       768       896         45.1  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ :dflash_t…  Push…  264  264         19        456    1,790,738,514  3,927,0…  3,927,2…  3,908,4…  3,949,6…      6,775.3  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  264  264         19      3,553    1,054,698,233  296,847…  290,248…   240,103  1,909,2…     50,856.6  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  264  264         19      3,496      123,435,530  35,307.6  32,480.5    13,441    70,722     11,620.1  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  264  264         19      1,748      104,960,236  60,045.9  59,985.5    58,082    73,538        912.8  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  264  264         19      3,990       94,463,295  23,675.0   6,464.0     4,064    49,986     20,168.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  264  264         19         19       62,610,155  3,295,2…  1,552,1…  1,381,9…  9,962,7…  3,061,572.6  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  264  264         19      1,311       52,652,032  40,161.7  40,161.0    39,553    40,961        234.5  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  264  264         19      3,078       44,685,152  14,517.6  18,128.5     5,280    24,097      7,301.6  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  264  264         19      3,648       23,273,775   6,379.9   6,240.0     4,480     8,192        638.6  _score_kernel                                                                                       
+ :dflash_t…  Push…  264  264         19      1,748       14,569,084   8,334.7   8,288.0     5,600    12,960      1,031.9  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  264  264         19        456       12,495,166  27,401.7  27,456.5    25,857    29,185        587.2  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  264  264         19      2,204        8,575,148   3,890.7   4,160.0     2,400     4,705        644.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  264  264         19      3,097        8,192,694   2,645.4   2,720.0     1,792     5,345        498.7  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  264  264         19      3,648        7,566,148   2,074.1   2,112.0       928     3,136        517.0  _combine_kernel                                                                                     
+ :dflash_t…  Push…  264  264         19      3,553        7,372,455   2,075.0   2,080.0     1,888     2,400         63.2  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  264  264         19      1,311        6,885,376   5,252.0   5,248.0     4,800     6,336        208.9  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  264  264         19      1,748        6,708,139   3,837.6   3,839.0     3,616     4,224         78.9  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  264  264         19      1,748        6,082,782   3,479.9   3,456.0     3,328     3,873         76.3  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  264  264         19      1,748        4,649,028   2,659.6   2,656.0     2,304     3,200        106.9  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  264  264         19      1,748        4,131,057   2,363.3   2,368.0     2,240     2,688         45.6  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  264  264         19      1,748        3,986,699   2,280.7   2,272.0     2,208     2,624         53.2  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  264  264         19      1,748        3,610,785   2,065.7   2,048.0     1,952     2,432         62.1  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  264  264         19      1,311        3,506,093   2,674.4   2,656.0     2,336     3,232         96.3  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  264  264         19      1,748        3,309,852   1,893.5   1,888.0     1,823     2,240         39.7  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  264  264         19      1,748        3,308,673   1,892.8   1,888.0     1,824     2,144         34.0  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  264  264         19      1,748        3,167,956   1,812.3   1,793.0     1,728     2,016         32.9  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  264  264         19      1,748        3,122,683   1,786.4   1,792.0     1,664     2,113         46.0  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  264  264         19      1,311        2,892,184   2,206.1   2,208.0     2,112     2,496         51.1  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  264  264         19      1,482        2,810,787   1,896.6   1,856.0     1,632     5,600        416.9  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  264  264         19        456        2,792,584   6,124.1   6,112.0     5,920     6,464         89.9  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  264  264         19      1,748        2,477,245   1,417.2   1,376.0     1,280     2,048         99.0  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  264  264         19      1,748        2,290,332   1,310.3   1,280.0     1,216     1,760         68.4  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  264  264         19        475        2,124,855   4,473.4   3,456.0     3,168    27,137      4,507.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  264  264         19      1,748        2,055,778   1,176.1   1,152.0     1,120     1,504         44.6  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  264  264         19      1,634        1,578,389     966.0     960.0       800     1,184         37.1  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  264  264         19        912        1,518,251   1,664.7   1,664.0     1,536     1,952         99.4  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  264  264         19        456        1,377,190   3,020.2   3,008.0     2,272     3,968        305.3  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  264  264         19         19        1,229,184  64,693.9  64,834.0    63,425    65,602        572.9  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  264  264         19        475          951,764   2,003.7   1,984.0     1,856     2,496        118.6  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  264  264         19        456          655,952   1,438.5   1,440.0     1,376     1,760         43.4  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  264  264         19        456          596,687   1,308.5   1,248.0     1,120     3,040        280.9  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  264  264         19         19          129,761   6,829.5   6,816.0     6,752     6,880         41.8  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  264  264         19         19           66,401   3,494.8   3,488.0     3,360     3,648         59.9  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  264  264         19         19           46,242   2,433.8   2,400.0     2,272     2,912        157.3  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  264  264         19         19           22,048   1,160.4   1,248.0       800     1,472        232.9  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  264  264         19         19           16,960     892.6     896.0       832       992         36.8  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  264  264         19         19           15,137     796.7     800.0       768       928         35.2  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ :dflash_t…  Push…  265  265         19        456    1,789,354,295  3,924,0…  3,923,5…  3,902,8…  3,948,0…      7,735.8  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  265  265         19      3,553    1,054,474,756  296,784…  287,368…   260,615  1,854,8…     48,107.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  265  265         19      3,496      123,657,209  35,371.1  32,673.0    13,216    73,090     11,615.0  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  265  265         19      1,748      105,015,992  60,077.8  60,002.0    58,306    71,042        857.0  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  265  265         19      3,990       94,289,909  23,631.6   6,401.0     4,096    50,178     20,142.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  265  265         19         19       62,995,528  3,315,5…  1,594,0…  1,401,0…  9,951,2…  3,055,786.8  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  265  265         19      1,311       53,184,542  40,567.9  40,577.0    39,937    41,377        229.0  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  265  265         19      3,078       45,042,612  14,633.7  18,736.0     5,345    23,617      7,338.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  265  265         19      3,648       23,146,486   6,345.0   6,272.0     4,512     8,064        574.1  _score_kernel                                                                                       
+ :dflash_t…  Push…  265  265         19      1,748       14,701,118   8,410.3   8,320.5     5,600    13,345      1,045.1  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  265  265         19        456       12,500,081  27,412.5  27,361.0    25,953    29,889        667.4  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  265  265         19      2,204        8,600,169   3,902.1   4,160.0     2,400     4,768        657.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  265  265         19      3,097        8,195,216   2,646.2   2,752.0     1,760     5,504        480.5  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  265  265         19      3,648        7,659,277   2,099.6   2,112.0       928     3,201        525.3  _combine_kernel                                                                                     
+ :dflash_t…  Push…  265  265         19      3,553        7,405,966   2,084.4   2,080.0     1,920     2,528         65.2  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  265  265         19      1,311        7,096,510   5,413.1   5,472.0     4,896     6,304        194.2  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  265  265         19      1,748        6,681,214   3,822.2   3,808.0     3,616     4,224         80.7  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  265  265         19      1,748        6,113,509   3,497.4   3,488.0     3,296     3,872         85.1  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  265  265         19      1,748        4,686,121   2,680.8   2,656.0     2,336     3,104        110.1  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  265  265         19      1,748        4,170,808   2,386.0   2,368.0     2,272     2,752         52.8  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  265  265         19      1,748        4,034,189   2,307.9   2,304.0     2,240     2,752         57.9  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  265  265         19      1,748        3,639,081   2,081.9   2,080.0     1,952     4,608         87.8  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  265  265         19      1,311        3,467,718   2,645.1   2,624.0     2,400     3,200         85.9  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  265  265         19      1,748        3,334,581   1,907.7   1,888.0     1,824     2,112         39.4  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  265  265         19      1,748        3,324,527   1,901.9   1,888.0     1,823     2,272         45.8  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  265  265         19      1,748        3,201,396   1,831.5   1,824.0     1,760     2,048         37.7  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  265  265         19      1,748        3,127,732   1,789.3   1,792.0     1,696     2,144         52.6  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  265  265         19      1,311        2,879,833   2,196.7   2,176.0     2,080     2,496         54.2  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  265  265         19      1,482        2,834,202   1,912.4   1,857.0     1,632     5,728        423.6  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  265  265         19        456        2,827,352   6,200.3   6,176.0     5,984     6,592        116.2  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  265  265         19      1,748        2,589,234   1,481.3   1,472.0     1,280     2,080        115.4  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  265  265         19      1,748        2,306,794   1,319.7   1,312.0     1,216     1,728         66.7  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  265  265         19        475        2,124,828   4,473.3   3,424.0     3,168    26,945      4,544.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  265  265         19      1,748        2,039,080   1,166.5   1,152.0     1,088     1,568         47.6  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  265  265         19      1,634        1,592,875     974.8     960.0       800     1,184         35.3  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  265  265         19        912        1,518,224   1,664.7   1,664.0     1,536     1,984         95.5  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  265  265         19        456        1,354,314   2,970.0   2,944.0     2,400     3,936        300.3  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  265  265         19         19        1,230,594  64,768.1  64,866.0    63,554    66,466        789.3  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  265  265         19        475          959,289   2,019.6   1,984.0     1,888     2,560        128.0  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  265  265         19        456          655,304   1,437.1   1,440.0     1,376     1,760         41.5  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  265  265         19        456          600,565   1,317.0   1,248.0     1,120     3,361        337.7  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  265  265         19         19          130,948   6,892.0   6,880.0     6,816     7,104         61.4  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  265  265         19         19           66,337   3,491.4   3,488.0     3,456     3,521         26.0  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  265  265         19         19           47,492   2,499.6   2,464.0     2,336     2,752        120.7  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  265  265         19         19           21,376   1,125.1     960.0       832     2,240        359.6  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  265  265         19         19           17,121     901.1     896.0       832     1,024         35.8  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  265  265         19         19           15,040     791.6     800.0       736       896         36.7  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ :dflash_t…  Push…  266  266         19        456    1,782,684,325  3,909,3…  3,909,3…  3,889,8…  3,929,3…      7,502.8  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  266  266         19      3,553    1,061,411,926  298,736…  286,921…   260,040  1,919,1…     49,661.0  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  266  266         19      3,496      123,839,851  35,423.3  32,721.0    13,633    71,778     11,595.8  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  266  266         19      1,748      105,112,787  60,133.2  60,066.0    57,954    72,322        950.1  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  266  266         19      3,990       94,586,619  23,705.9   6,432.0     4,064    49,857     20,208.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  266  266         19         19       63,010,946  3,316,3…  1,590,7…  1,406,7…  9,954,7…  3,052,598.1  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  266  266         19      1,311       52,914,837  40,362.2  40,353.0    39,873    41,121        234.5  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  266  266         19      3,078       45,108,898  14,655.3  18,880.0     5,184    27,009      7,360.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  266  266         19      3,648       23,225,811   6,366.7   6,272.0     4,480     7,904        595.8  _score_kernel                                                                                       
+ :dflash_t…  Push…  266  266         19      1,748       14,483,518   8,285.8   8,225.0     5,440    13,184      1,029.4  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  266  266         19        456       12,516,597  27,448.7  27,489.0    25,761    28,801        565.1  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  266  266         19      2,204        8,594,936   3,899.7   4,160.0     2,432     4,737        657.0  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  266  266         19      3,097        8,229,240   2,657.2   2,752.0     1,760     5,408        489.5  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  266  266         19      3,648        7,694,929   2,109.4   2,144.0       960     3,200        533.4  _combine_kernel                                                                                     
+ :dflash_t…  Push…  266  266         19      3,553        7,373,739   2,075.4   2,080.0     1,920     2,496         67.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  266  266         19      1,311        6,812,660   5,196.5   5,152.0     4,801     6,752        225.6  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  266  266         19      1,748        6,791,266   3,885.2   3,872.0     3,648     4,288         78.6  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  266  266         19      1,748        6,272,355   3,588.3   3,584.0     3,424     3,969         82.0  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  266  266         19      1,748        4,680,946   2,677.9   2,656.0     2,368     3,200        107.9  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  266  266         19      1,748        4,174,504   2,388.2   2,369.0     2,304     2,688         46.6  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  266  266         19      1,748        4,068,985   2,327.8   2,304.0     2,240     2,656         50.6  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  266  266         19      1,748        3,640,311   2,082.6   2,080.0     1,920     2,432         63.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  266  266         19      1,311        3,497,581   2,667.9   2,656.0     2,336     3,264         89.5  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  266  266         19      1,748        3,353,984   1,918.8   1,920.0     1,824     2,176         39.5  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  266  266         19      1,748        3,279,310   1,876.0   1,856.0     1,791     2,176         42.7  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  266  266         19      1,748        3,213,171   1,838.2   1,824.0     1,792     2,048         32.1  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  266  266         19      1,748        3,124,382   1,787.4   1,792.0     1,696     2,048         44.0  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  266  266         19      1,482        2,886,101   1,947.4   1,920.0     1,696     5,632        415.7  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  266  266         19      1,311        2,839,384   2,165.8   2,144.0     2,080     2,496         51.2  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  266  266         19        456        2,764,597   6,062.7   6,048.0     5,856     6,464        108.2  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  266  266         19      1,748        2,510,145   1,436.0   1,408.0     1,280     2,016        104.0  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  266  266         19      1,748        2,302,368   1,317.1   1,312.0     1,216     1,664         61.9  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  266  266         19        475        2,128,872   4,481.8   3,456.0     3,200    27,457      4,552.7  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  266  266         19      1,748        2,005,764   1,147.5   1,152.0     1,088     1,408         42.8  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  266  266         19      1,634        1,601,594     980.2     992.0       800     1,216         36.1  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  266  266         19        912        1,499,824   1,644.5   1,632.0     1,504     1,952         95.1  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  266  266         19        456        1,354,544   2,970.5   2,944.0     2,240     4,864        302.8  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  266  266         19         19        1,239,687  65,246.7  65,186.0    64,034    66,530        697.3  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  266  266         19        475          966,869   2,035.5   1,984.0     1,920     2,593        124.8  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  266  266         19        456          640,437   1,404.5   1,408.0     1,344     1,600         30.4  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  266  266         19        456          589,208   1,292.1   1,216.0     1,088     3,584        322.5  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  266  266         19         19          130,214   6,853.4   6,848.0     6,784     7,008         56.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  266  266         19         19           66,945   3,523.4   3,520.0     3,392     3,584         42.5  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  266  266         19         19           48,865   2,571.8   2,528.0     2,368     3,072        174.7  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  266  266         19         19           23,137   1,217.7   1,312.0       896     1,568        235.3  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  266  266         19         19           17,248     907.8     896.0       896       928         15.9  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  266  266         19         19           14,625     769.7     768.0       736       832         19.9  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ :dflash_t…  Push…  267  267         19        456    1,763,015,939  3,866,2…  3,866,7…  3,847,2…  3,892,9…      6,714.8  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  267  267         19      3,553    1,084,170,149  305,142…  286,922…   224,999  1,966,8…     56,108.5  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  267  267         19      3,496      123,523,953  35,332.9  32,417.0    13,601    72,258     11,599.9  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  267  267         19      1,748      105,353,209  60,270.7  60,194.0    57,794    73,187        979.6  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  267  267         19      3,990       94,496,877  23,683.4   6,400.0     4,032    50,209     20,233.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  267  267         19         19       63,136,141  3,322,9…  1,670,4…  1,408,8…  9,980,5…  3,053,583.7  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  267  267         19      1,311       52,172,537  39,796.0  39,809.0    39,105    40,961        279.8  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  267  267         19      3,078       44,969,418  14,609.9  19,040.5     5,248    23,777      7,338.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  267  267         19      3,648       23,089,540   6,329.4   6,208.0     4,449     7,840        623.8  _score_kernel                                                                                       
+ :dflash_t…  Push…  267  267         19      1,748       14,363,242   8,217.0   8,161.0     5,472    12,993      1,017.4  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  267  267         19        456       12,505,545  27,424.4  27,425.0    25,953    28,993        571.6  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  267  267         19      2,204        8,537,923   3,873.8   4,128.0     2,400     4,800        642.9  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  267  267         19      3,097        8,121,380   2,622.3   2,720.0     1,760     5,344        482.7  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  267  267         19      3,648        7,614,321   2,087.3   2,112.0       928     3,200        523.8  _combine_kernel                                                                                     
+ :dflash_t…  Push…  267  267         19      3,553        7,330,216   2,063.1   2,048.0     1,888     2,432         68.1  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  267  267         19      1,311        6,997,213   5,337.3   5,376.0     4,768     6,241        210.6  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  267  267         19      1,748        6,734,266   3,852.6   3,840.0     3,647     4,320         77.7  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  267  267         19      1,748        6,098,813   3,489.0   3,488.0     3,296     3,872         82.6  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  267  267         19      1,748        4,602,366   2,632.9   2,624.0     2,336     3,136        107.8  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  267  267         19      1,748        4,089,897   2,339.8   2,336.0     2,240     2,656         48.0  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  267  267         19      1,748        3,974,606   2,273.8   2,272.0     2,208     2,624         56.2  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  267  267         19      1,748        3,574,135   2,044.7   2,017.0     1,920     3,488         75.7  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  267  267         19      1,311        3,474,831   2,650.5   2,624.0     2,304     3,456         97.7  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  267  267         19      1,748        3,316,558   1,897.3   1,888.0     1,792     2,144         39.9  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  267  267         19      1,748        3,261,334   1,865.8   1,856.0     1,791     2,240         46.5  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  267  267         19      1,748        3,167,790   1,812.2   1,792.0     1,728     2,048         37.3  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  267  267         19      1,748        3,089,649   1,767.5   1,760.0     1,632     2,016         46.9  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  267  267         19      1,311        2,822,016   2,152.6   2,144.0     2,048     2,432         52.4  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  267  267         19      1,482        2,820,825   1,903.4   1,856.0     1,632     5,537        414.6  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  267  267         19        456        2,770,633   6,075.9   6,049.0     5,824     6,369        105.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  267  267         19      1,748        2,534,064   1,449.7   1,440.0     1,280     2,048        110.5  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  267  267         19      1,748        2,269,985   1,298.6   1,280.0     1,184     1,632         62.7  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  267  267         19        475        2,104,907   4,431.4   3,424.0     3,168    26,977      4,547.6  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  267  267         19      1,748        1,988,442   1,137.6   1,120.0     1,088     1,472         42.4  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  267  267         19      1,634        1,571,605     961.8     960.0       800     1,152         34.2  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  267  267         19        912        1,511,920   1,657.8   1,648.5     1,504     1,984        100.8  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  267  267         19        456        1,332,520   2,922.2   2,880.0     2,336     4,032        288.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  267  267         19         19        1,232,139  64,849.4  64,834.0    63,971    66,083        624.7  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  267  267         19        475          956,192   2,013.0   1,984.0     1,888     2,624        119.4  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  267  267         19        456          639,824   1,403.1   1,408.0     1,344     1,728         38.2  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  267  267         19        456          588,597   1,290.8   1,216.0     1,088     3,360        266.0  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  267  267         19         19          129,987   6,841.4   6,848.0     6,784     6,944         40.7  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  267  267         19         19           66,433   3,496.5   3,488.0     3,392     3,553         39.8  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  267  267         19         19           47,265   2,487.6   2,464.0     2,336     2,721        124.9  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  267  267         19         19           19,712   1,037.5     960.0       800     2,048        319.7  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  267  267         19         19           17,568     924.6     896.0       864     1,056         53.2  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  267  267         19         19           14,592     768.0     768.0       736       896         33.7  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ :dflash_t…  Push…  268  268         19        456    1,766,208,950  3,873,2…  3,872,2…  3,854,1…  3,904,1…      8,935.7  _fwd_kernel                                                                                         
+ :dflash_t…  Push…  268  268         19      3,553    1,082,442,207  304,655…  300,264…   225,190  1,897,7…     56,428.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ :dflash_t…  Push…  268  268         19      3,496      123,488,797  35,322.9  32,513.0    12,864    70,530     11,551.5  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+ :dflash_t…  Push…  268  268         19      1,748      104,911,006  60,017.7  59,938.0    58,017    68,130        851.0  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+ :dflash_t…  Push…  268  268         19      3,990       94,301,890  23,634.6   6,369.0     4,032    50,722     20,178.5  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+ :dflash_t…  Push…  268  268         19         19       63,644,583  3,349,7…  1,710,4…  1,429,6…  10,032,…  3,050,186.9  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+ :dflash_t…  Push…  268  268         19      1,311       52,134,236  39,766.8  39,745.0    38,946    40,802        308.1  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+ :dflash_t…  Push…  268  268         19      3,078       45,095,450  14,650.9  18,657.0     5,152    25,921      7,381.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+ :dflash_t…  Push…  268  268         19      3,648       22,889,158   6,274.4   6,240.0     4,481     7,744        546.9  _score_kernel                                                                                       
+ :dflash_t…  Push…  268  268         19      1,748       14,350,293   8,209.5   8,192.0     5,824    12,832      1,017.1  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+ :dflash_t…  Push…  268  268         19        456       12,557,642  27,538.7  27,536.5    25,984    29,472        602.2  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+ :dflash_t…  Push…  268  268         19      2,204        8,535,158   3,872.6   4,128.0     2,400     4,704        649.7  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+ :dflash_t…  Push…  268  268         19      3,097        8,134,000   2,626.4   2,689.0     1,760     5,184        494.5  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+ :dflash_t…  Push…  268  268         19      3,648        7,536,771   2,066.0   2,080.0       896     3,040        520.4  _combine_kernel                                                                                     
+ :dflash_t…  Push…  268  268         19      3,553        7,351,510   2,069.1   2,048.0     1,888     2,464         70.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  268  268         19      1,311        7,008,368   5,345.8   5,408.0     4,736     6,784        199.1  _causal_conv1d_update_kernel                                                                        
+ :dflash_t…  Push…  268  268         19      1,748        6,726,601   3,848.2   3,840.0     3,647     4,863         81.6  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+ :dflash_t…  Push…  268  268         19      1,748        6,062,874   3,468.5   3,456.0     3,296     3,936         85.2  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+ :dflash_t…  Push…  268  268         19      1,748        4,613,628   2,639.4   2,624.0     2,304     3,073        109.6  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+ :dflash_t…  Push…  268  268         19      1,748        4,095,255   2,342.8   2,336.0     2,240     2,656         53.0  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+ :dflash_t…  Push…  268  268         19      1,748        3,965,994   2,268.9   2,240.0     2,176     2,624         59.8  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+ :dflash_t…  Push…  268  268         19      1,748        3,592,170   2,055.0   2,048.0     1,920     2,369         65.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+ :dflash_t…  Push…  268  268         19      1,311        3,455,834   2,636.0   2,624.0     2,368     3,232         86.3  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+ :dflash_t…  Push…  268  268         19      1,748        3,294,790   1,884.9   1,888.0     1,792     2,112         38.7  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  268  268         19      1,748        3,286,118   1,879.9   1,856.0     1,823     2,240         47.1  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+ :dflash_t…  Push…  268  268         19      1,748        3,168,506   1,812.6   1,793.0     1,728     2,016         34.1  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+ :dflash_t…  Push…  268  268         19      1,748        3,106,742   1,777.3   1,760.0     1,664     2,112         52.3  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+ :dflash_t…  Push…  268  268         19      1,311        2,868,096   2,187.7   2,176.0     2,080     2,496         51.7  layer_norm_gated_fwd_kernel                                                                         
+ :dflash_t…  Push…  268  268         19        456        2,809,586   6,161.4   6,144.0     5,984     6,560        101.1  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+ :dflash_t…  Push…  268  268         19      1,482        2,783,159   1,878.0   1,824.0     1,600     5,568        418.7  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+ :dflash_t…  Push…  268  268         19      1,748        2,438,092   1,394.8   1,344.0     1,280     1,856         94.7  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+ :dflash_t…  Push…  268  268         19      1,748        2,248,739   1,286.5   1,280.0     1,184     1,664         67.8  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+ :dflash_t…  Push…  268  268         19        475        2,122,197   4,467.8   3,456.0     3,200    27,360      4,552.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+ :dflash_t…  Push…  268  268         19      1,748        2,056,253   1,176.3   1,152.0     1,088     1,504         50.1  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+ :dflash_t…  Push…  268  268         19      1,634        1,566,023     958.4     960.0       800     1,152         35.5  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+ :dflash_t…  Push…  268  268         19        912        1,520,404   1,667.1   1,664.5     1,504     2,048        100.0  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+ :dflash_t…  Push…  268  268         19        456        1,355,857   2,973.4   2,976.0     2,304     3,872        283.9  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  268  268         19         19        1,239,109  65,216.3  65,058.0    64,258    66,914        704.2  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+ :dflash_t…  Push…  268  268         19        475          947,547   1,994.8   1,952.0     1,888     2,497        115.4  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+ :dflash_t…  Push…  268  268         19        456          645,334   1,415.2   1,408.0     1,344     1,696         47.4  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+ :dflash_t…  Push…  268  268         19        456          596,588   1,308.3   1,248.0     1,088     3,488        305.0  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+ :dflash_t…  Push…  268  268         19         19          129,733   6,828.1   6,816.0     6,721     6,945         54.6  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+ :dflash_t…  Push…  268  268         19         19           66,049   3,476.3   3,488.0     3,424     3,552         34.2  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+ :dflash_t…  Push…  268  268         19         19           45,153   2,376.5   2,336.0     2,272     2,592         89.8  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+ :dflash_t…  Push…  268  268         19         19           17,184     904.4     928.0       800       960         58.3  _vocab_parallel_embedding_kernel                                                                    
+ :dflash_t…  Push…  268  268         19         19           17,089     899.4     896.0       864       992         25.9  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+ :dflash_t…  Push…  268  268         19         19           15,008     789.9     800.0       768       832         24.0  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+ CCCL:cub:…  Push…  261  261          1          4           20,257   5,064.3   4,864.5     4,768     5,760        468.9  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  261  261          1          1            2,272   2,272.0   2,272.0     2,272     2,272          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  261  261          1          1              993     993.0     993.0       993       993          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  262  262          1          4           20,224   5,056.0   4,864.0     4,768     5,728        453.3  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  262  262          1          1            2,272   2,272.0   2,272.0     2,272     2,272          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  262  262          1          1              992     992.0     992.0       992       992          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  263  263          1          4           20,225   5,056.3   4,848.0     4,768     5,761        473.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  263  263          1          1            2,272   2,272.0   2,272.0     2,272     2,272          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  263  263          1          1            1,024   1,024.0   1,024.0     1,024     1,024          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  264  264          1          4           20,481   5,120.3   4,880.5     4,832     5,888        513.8  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  264  264          1          1            2,368   2,368.0   2,368.0     2,368     2,368          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  264  264          1          1            1,024   1,024.0   1,024.0     1,024     1,024          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  265  265          1          4           20,481   5,120.3   4,928.0     4,832     5,793        453.8  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  265  265          1          1            2,272   2,272.0   2,272.0     2,272     2,272          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  265  265          1          1            1,024   1,024.0   1,024.0     1,024     1,024          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  266  266          1          4           20,320   5,080.0   4,896.0     4,800     5,728        437.5  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  266  266          1          1            2,272   2,272.0   2,272.0     2,272     2,272          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  266  266          1          1              992     992.0     992.0       992       992          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  267  267          1          4           19,713   4,928.3   4,736.0     4,640     5,601        453.8  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  267  267          1          1            2,240   2,240.0   2,240.0     2,240     2,240          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  267  267          1          1            1,120   1,120.0   1,120.0     1,120     1,120          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  268  268          1          4           20,065   5,016.3   4,832.0     4,705     5,696        463.8  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+ CCCL:cub:…  Push…  268  268          1          1            2,272   2,272.0   2,272.0     2,272     2,272          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+ CCCL:cub:…  Push…  268  268          1          1            1,056   1,056.0   1,056.0     1,056     1,056          0.0  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+ CCCL:cub:…  Push…  261  261        100        100          125,349   1,253.5   1,056.0       992     1,697        266.8  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  261  261        100        100           61,217     612.2     544.0       512       864        114.8  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  262  262         95         95          117,410   1,235.9   1,056.0       992     1,632        256.7  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  262  262         95         95           58,785     618.8     544.0       512       992        126.6  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  263  263         95         95          118,469   1,247.0   1,056.0       960     1,696        259.2  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  263  263         95         95           58,146     612.1     544.0       512       928        119.3  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  264  264         95         95          120,993   1,273.6   1,056.0     1,024     1,696        266.5  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  264  264         95         95           59,939     630.9     544.0       512     1,024        124.2  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  265  265         95         95          120,324   1,266.6   1,056.0     1,024     1,824        268.7  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  265  265         95         95           58,849     619.5     544.0       512       896        124.8  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  266  266         95         95          117,379   1,235.6   1,056.0       992     1,696        258.5  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  266  266         95         95           58,051     611.1     512.0       512       800        120.1  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  267  267         95         95          116,835   1,229.8   1,056.0       960     1,696        262.7  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  267  267         95         95           58,114     611.7     544.0       480       896        120.5  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  268  268         95         95          120,228   1,265.6   1,056.0       992     1,696        273.9  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+ CCCL:cub:…  Push…  268  268         95         95           59,074     621.8     544.0       512       992        122.4  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+ CCCL:cub:…  Push…  261  261          1          1            4,192   4,192.0   4,192.0     4,192     4,192          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  261  261          1          1              704     704.0     704.0       704       704          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ CCCL:cub:…  Push…  262  262          1          1            3,840   3,840.0   3,840.0     3,840     3,840          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  262  262          1          1              768     768.0     768.0       768       768          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ CCCL:cub:…  Push…  263  263          1          1            3,712   3,712.0   3,712.0     3,712     3,712          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  263  263          1          1              768     768.0     768.0       768       768          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ CCCL:cub:…  Push…  264  264          1          1            4,064   4,064.0   4,064.0     4,064     4,064          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  264  264          1          1              736     736.0     736.0       736       736          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ CCCL:cub:…  Push…  265  265          1          1            3,808   3,808.0   3,808.0     3,808     3,808          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  265  265          1          1              736     736.0     736.0       736       736          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ CCCL:cub:…  Push…  266  266          1          1            3,904   3,904.0   3,904.0     3,904     3,904          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  266  266          1          1              704     704.0     704.0       704       704          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ CCCL:cub:…  Push…  267  267          1          1            3,969   3,969.0   3,969.0     3,969     3,969          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  267  267          1          1              736     736.0     736.0       736       736          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ CCCL:cub:…  Push…  268  268          1          1            4,256   4,256.0   4,256.0     4,256     4,256          0.0  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+ CCCL:cub:…  Push…  268  268          1          1              736     736.0     736.0       736       736          0.0  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+ NCCL:API …  Star…  261  261         20         20       16,619,589  830,979…  349,562…   216,455  9,565,1…  2,059,162.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:API …  Star…  262  262         19         19       18,889,577  994,188…  340,938…   226,502  12,461,…  2,778,620.5  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:API …  Star…  263  263         19         19       17,480,018  920,000…  343,658…   226,983  11,108,…  2,468,905.9  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:API …  Star…  264  264         19         19       16,614,565  874,450…  334,730…   244,231  10,379,…  2,303,450.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:API …  Star…  265  265         19         19       16,448,571  865,714…  311,433…   262,247  10,613,…  2,362,380.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:API …  Star…  266  266         19         19       14,995,881  789,256…  306,282…   262,472  9,189,2…  2,035,563.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:API …  Star…  267  267         19         19       12,042,135  633,796…  304,330…   264,617  6,266,6…  1,365,332.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:API …  Star…  268  268         19         19       15,079,283  793,646…  261,608…   224,839  9,994,3…  2,228,833.7  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  261  261         20         20       16,619,589  830,979…  349,562…   216,455  9,565,1…  2,059,162.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  262  262         19         19       18,889,577  994,188…  340,938…   226,502  12,461,…  2,778,620.5  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  263  263         19         19       17,480,018  920,000…  343,658…   226,983  11,108,…  2,468,905.9  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  264  264         19         19       16,614,565  874,450…  334,730…   244,231  10,379,…  2,303,450.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  265  265         19         19       16,448,571  865,714…  311,433…   262,247  10,613,…  2,362,380.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  266  266         19         19       14,995,881  789,256…  306,282…   262,472  9,189,2…  2,035,563.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  267  267         19         19       12,042,135  633,796…  304,330…   264,617  6,266,6…  1,365,332.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Grou…  Star…  268  268         19         19       15,079,283  793,646…  261,608…   224,839  9,994,3…  2,228,833.7  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  261  261         20         20       16,619,589  830,979…  349,562…   216,455  9,565,1…  2,059,162.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  262  262         19         19       18,889,577  994,188…  340,938…   226,502  12,461,…  2,778,620.5  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  263  263         19         19       17,480,018  920,000…  343,658…   226,983  11,108,…  2,468,905.9  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  264  264         19         19       16,614,565  874,450…  334,730…   244,231  10,379,…  2,303,450.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  265  265         19         19       16,448,571  865,714…  311,433…   262,247  10,613,…  2,362,380.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  266  266         19         19       14,995,881  789,256…  306,282…   262,472  9,189,2…  2,035,563.2  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  267  267         19         19       12,042,135  633,796…  304,330…   264,617  6,266,6…  1,365,332.1  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+ NCCL:Kern…  Star…  268  268         19         19       15,079,283  793,646…  261,608…   224,839  9,994,3…  2,228,833.7  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/nsys_stats.txt

@@ -0,0 +1,174 @@
+
+NOTICE: Existing SQLite export found: /report/profiles/node_5/nsys/d0.sqlite
+        It is assumed file was previously exported from: /report/profiles/node_5/nsys/d0.nsys-rep
+        Consider using --force-export=true if needed.
+
+Processing [/report/profiles/node_5/nsys/d0.sqlite] with [/opt/nvidia/nsight-systems-cli/2026.4.1/target-linux-x64/reports/cuda_gpu_kern_sum.py]... 
+
+ ** CUDA GPU Kernel Summary (cuda_gpu_kern_sum):
+
+ Time (%)  Total Time (ns)  Instances   Avg (ns)     Med (ns)    Min (ns)   Max (ns)   StdDev (ns)                                                  Name                                                
+ --------  ---------------  ---------  -----------  -----------  --------  ----------  -----------  ----------------------------------------------------------------------------------------------------
+     49.8   14,590,525,348      4,590  3,178,763.7  3,891,418.0   284,937   3,949,675  1,438,067.1  _fwd_kernel                                                                                         
+     31.9    9,340,217,936     30,600    305,235.9    288,426.0   215,750  52,394,336    348,000.6  ncclDevKernel_AllReduce_Sum_bf16_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)             
+      3.4      995,286,483     28,152     35,354.0     32,577.0    12,864      73,090     11,589.5  void cutlass::device_kernel<cutlass::gemm::kernel::GemmUniversal<cutlass::gemm::GroupProblemShape<c…
+      2.9      846,418,308     14,076     60,132.0     60,066.0    57,634      73,538        928.6  void cutlass::Kernel2<cutlass_80_tensorop_s16816gemm_bf16_256x64_32x4_tn_align8>(T1::Params)        
+      2.6      767,151,972     33,048     23,213.3      6,433.0     4,000      50,722     20,092.6  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_16x16_128x2_tn_align8>(T1::Par…
+      1.8      532,711,449        459  1,160,591.4     89,891.0    53,761  10,032,449  2,298,458.7  ncclDevKernel_AllGather_RING_LL(ncclDevKernelArgsStorage<(unsigned long)4096>)                      
+      1.4      423,285,388     10,557     40,095.2     40,129.0    38,850      41,377        418.0  fused_sigmoid_gating_delta_rule_update_kernel                                                       
+      1.3      379,086,218     26,622     14,239.6     17,761.0     5,152      27,009      7,252.8  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_tn_align8>(T1::Para…
+      0.6      186,309,304     29,376      6,342.2      6,240.0     4,416       8,192        606.2  _score_kernel                                                                                       
+      0.4      116,686,329     14,076      8,289.7      8,225.0     5,409      13,345      1,028.3  void tensorrt_llm::kernels::cutlass_kernels::finalizeMoeRoutingKernel<__nv_bfloat16, __nv_bfloat16,…
+      0.3      100,782,270      3,672     27,446.2     27,457.0    25,761      29,889        586.0  kernel_cutlass__dsv3_fused_a_gemm_kernel_tensorptri32gmemo2112358435841_tensorptri32gmemo1635843584…
+      0.3       78,720,700        153    514,514.4    514,447.0   512,207     531,088      1,661.3  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_128x1_tn_align8>(T1::Par…
+      0.2       71,900,977     18,666      3,852.0      4,128.0     2,368       4,832        653.2  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Para…
+      0.2       71,014,349     27,234      2,607.6      2,720.0     1,568       5,504        503.4  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, __nv_bfloat16, __nv_bfloat16, float, __nv…
+      0.2       61,252,462     29,376      2,085.1      2,112.0       896       3,328        522.9  _combine_kernel                                                                                     
+      0.2       60,376,010     28,917      2,087.9      2,049.0     1,888       5,952        216.5  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+      0.2       56,038,934     10,557      5,308.2      5,344.0     4,704       6,784        219.6  _causal_conv1d_update_kernel                                                                        
+      0.2       54,128,829     14,076      3,845.5      3,840.0     3,616       4,863         81.1  void tensorrt_llm::kernels::cutlass_kernels::doActivationKernel<__nv_fp8_e4m3, __nv_bfloat16, __nv_…
+      0.2       49,309,425     14,076      3,503.1      3,488.0     3,296       3,969         88.6  void tensorrt_llm::kernels::cutlass_kernels::expandInputRowsKernel<__nv_fp8_e4m3, __nv_fp8_e4m3, (t…
+      0.1       37,420,001     14,076      2,658.4      2,656.0     2,304       3,233        110.0  void sglang::route_quant_fused_kernel<(bool)1, float, __nv_bfloat16>(sglang::RouteQuantFusedParams) 
+      0.1       33,274,101     14,076      2,363.9      2,368.0     2,208       2,784         53.7  void tensorrt_llm::kernels::cutlass_kernels::blockExpertPrefixSumKernel<(int)256>(const int *, int …
+      0.1       32,261,405     14,076      2,291.9      2,272.0     2,176       2,752         58.5  void tensorrt_llm::kernels::cutlass_kernels::computeStridesTmaWarpSpecializedKernel<__nv_fp8_e4m3, …
+      0.1       29,070,517     14,076      2,065.3      2,048.0     1,920       4,608         70.8  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign12…
+      0.1       27,968,788     10,557      2,649.3      2,624.0     2,304       3,456         92.2  void gemmSN_TN_kernel<float, (int)128, (int)16, (int)2, (int)4, (int)8, (int)9, (bool)0, cublasGemv…
+      0.1       26,748,027     14,076      1,900.3      1,888.0     1,792       2,208         39.8  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+      0.1       26,510,955     14,076      1,883.4      1,887.0     1,791       2,272         45.4  void tensorrt_llm::kernels::quantize_with_block_size<(tensorrt_llm::BlockScaleQuantizationType)2, _…
+      0.1       25,641,069     13,770      1,862.1      1,856.0     1,408       5,728        409.2  void at::native::elementwise_kernel<(int)128, (int)4, void at::native::gpu_kernel_impl_nocast<at::n…
+      0.1       25,617,459     14,076      1,819.9      1,824.0     1,728       2,049         36.0  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, float, float, float, (bool)0, floa…
+      0.1       25,021,158     14,076      1,777.6      1,760.0     1,632       2,144         50.1  void sglang::situ_and_mul_kernel<float, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMulParams)  
+      0.1       23,003,024     10,557      2,178.9      2,176.0     2,048       2,497         56.6  layer_norm_gated_fwd_kernel                                                                         
+      0.1       22,467,866      3,672      6,118.7      6,112.0     5,824       6,592        113.9  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+      0.1       20,225,634     14,076      1,436.9      1,408.0     1,248       2,080        109.1  void sglang::add3_kernel<(bool)1, (bool)1>(sglang::Add3Params)                                      
+      0.1       19,553,652        306     63,900.8     63,922.5    61,858      79,522      1,506.0  void cutlass::Kernel2<cutlass_80_tensorop_bf16_s16816gemm_relu_bf16_256x64_32x4_tn_align8>(T1::Para…
+      0.1       18,358,363     14,076      1,304.2      1,280.0     1,184       1,760         66.8  tensorrt_llm::kernels::cutlass_kernels::mergeExpertPrefixSumKernel(const int *, const int *, const …
+      0.1       17,067,934      3,825      4,462.2      3,456.0     3,104      27,457      4,532.6  void cutlass::Kernel2<cutlass_80_wmma_tensorop_s161616gemm_bf16_32x32_64x1_tn_align8>(T1::Params)   
+      0.1       16,291,644     14,076      1,157.4      1,152.0     1,056       1,568         46.6  void tensorrt_llm::kernels::cutlass_kernels::globalExpertPrefixSumKernel<(int)256>(const int *, int…
+      0.0       12,752,121     13,158        969.2        960.0       768       1,376         36.9  void at::native::vectorized_elementwise_kernel<(int)4, at::native::CUDAFunctor_add<c10::BFloat16>, …
+      0.0       12,160,760      7,344      1,655.9      1,648.5     1,472       2,048         98.9  void at::native::<unnamed>::CatArrayBatchedCopy<at::native::<unnamed>::OpaqueType<(unsigned int)2>,…
+      0.0       10,939,300      3,672      2,979.1      2,944.0     2,048       4,864        297.0  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+      0.0        7,683,600      3,825      2,008.8      1,984.0     1,856       2,624        119.9  void cublasLt::splitKreduce_kernel<(int)32, (int)16, int, float, __nv_bfloat16, float, __nv_bfloat1…
+      0.0        5,207,022      3,672      1,418.0      1,408.0     1,344       1,760         42.9  void sglang::mla_output_gate_kernel<(int)256, (bool)1>(sglang::MlaOutputGateParams)                 
+      0.0        5,069,771        459     11,045.3     15,041.0     2,816      16,864      5,774.6  create_flashinfer_kv_indices_triton                                                                 
+      0.0        4,776,945      3,672      1,300.9      1,248.0     1,088       3,584        292.6  kernel_cutlass_kernel_flashinfernormkernelsrmsnormRMSNormKernel_object_at__tensorptrbf16gmemalign16…
+      0.0        4,260,889      1,836      2,320.7      2,304.0     2,144       2,656         58.7  kernel_cutlass_kernel_flashinfernormkernelsfused_add_rmsnormFusedAddRMSNormKernel_object_at__tensor…
+      0.0        3,022,079        153     19,752.2     19,744.0    18,944      21,505        331.4  void cutlass::Kernel2<cutlass_80_wmma_tensorop_bf16_s161616gemm_bf16_32x32_64x2_nn_align8>(T1::Para…
+      0.0        2,772,279        918      3,019.9      3,040.0     2,784       3,137         51.1  void sglang::act_and_mul_kernel<__nv_bfloat16, (sglang::ActivationKind)0, (bool)1, (bool)0, (bool)0…
+      0.0        2,164,385        153     14,146.3     14,144.0    13,217      17,088        446.0  _fused_mamba_state_scatter_with_mask_kernel                                                         
+      0.0        1,458,665        153      9,533.8      9,536.0     9,312       9,792         87.5  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ma…
+      0.0        1,405,361        320      4,391.8      2,464.0     1,280      12,000      2,715.6  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+      0.0        1,230,757        153      8,044.2      8,032.0     7,872       8,384         95.2  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<float, at::native::ArgMaxOps<…
+      0.0        1,139,234        918      1,241.0      1,216.0     1,120       1,728         80.8  _table_qk_norm_rope_kernel                                                                          
+      0.0        1,050,014        153      6,862.8      6,849.0     6,688       7,104         64.1  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+      0.0        1,022,938        918      1,114.3      1,088.0     1,024       1,888        126.9  void sglang::store_kvcache<(long)256, (long)256, (int)1, (bool)1, long>(sglang::StoreKVCacheParams) 
+      0.0          956,987        765      1,251.0      1,056.0       960       1,824        263.4  void at_cuda_detail::cub::detail::scan::DeviceScanKernel<at_cuda_detail::cub::detail::scan::policy_…
+      0.0          949,915        918      1,034.8        944.5       832       1,664        224.3  set_kv_buffer_prefix_valid_tiled                                                                    
+      0.0          919,532        893      1,029.7      1,088.0       736       1,697        178.4  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+      0.0          754,770        322      2,344.0      2,336.0     1,312       3,009        308.7  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+      0.0          699,890        483      1,449.0      1,440.0     1,184       2,176        126.0  void at::native::unrolled_elementwise_kernel<at::native::direct_copy_kernel_cuda(at::TensorIterator…
+      0.0          623,732        592      1,053.6      1,056.0       832       1,472        129.7  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctorOnSelf_add<long>, std…
+      0.0          603,445        153      3,944.1      3,904.0     3,520       5,089        249.4  _fused_conv_window_scatter_multi_kernel                                                             
+      0.0          575,030        153      3,758.4      3,744.0     3,648       4,192         88.2  void at::native::reduce_kernel<(int)512, (int)1, at::native::ReduceOp<c10::BFloat16, at::native::Ar…
+      0.0          535,246        153      3,498.3      3,488.0     3,360       3,648         43.9  void sglang::situ_and_mul_kernel<__nv_bfloat16, __nv_bfloat16, (bool)1, (bool)1>(sglang::SituAndMul…
+      0.0          472,175        765        617.2        544.0       480       1,024        121.2  void at_cuda_detail::cub::detail::scan::DeviceScanInitKernel<at_cuda_detail::cub::ScanTileState<lon…
+      0.0          428,936        306      1,401.8      1,408.0     1,216       1,664         98.8  void at::native::<unnamed>::multi_tensor_apply_kernel<at::native::<unnamed>::TensorListMetadata<(in…
+      0.0          378,666        153      2,474.9      2,432.0     2,272       3,072        143.2  void at::native::<unnamed>::CatArrayBatchedCopy_vectorized<at::native::<unnamed>::OpaqueType<(unsig…
+      0.0          370,094        153      2,418.9      2,432.0     2,240       2,656         81.6  _dflash_accept_bonus_contig_kernel                                                                  
+      0.0          348,362         80      4,354.5      4,832.0     1,344       5,728      1,448.7  alloc_extend_kernel                                                                                 
+      0.0          332,681        313      1,062.9        928.0       800       2,240        239.5  _vocab_parallel_embedding_kernel                                                                    
+      0.0          321,512        306      1,050.7      1,056.0       992       1,344         46.1  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+      0.0          261,832        153      1,711.3      1,696.0     1,600       2,048         75.2  _fused_norm_rope_kernel_stacked                                                                     
+      0.0          248,327        153      1,623.1      1,600.0     1,536       1,856         63.0  void at::native::_scatter_gather_elementwise_kernel<(int)128, (int)8, void at::native::_cuda_scatte…
+      0.0          226,628        160      1,416.4      1,408.0     1,152       1,664         90.9  _prepare_dflash_draft_block_contig_kernel                                                           
+      0.0          216,363        306        707.1        704.0       640         928         37.1  void <unnamed>::elementwise_kernel_with_index<int, at::native::arange_cuda_out(const c10::Scalar &,…
+      0.0          181,541        153      1,186.5      1,184.0     1,120       1,344         43.2  _fused_replay_state_indices_kernel                                                                  
+      0.0          161,766         32      5,055.2      4,848.5     4,640       5,888        413.5  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortOnesweepKernel<at_cuda_detail::cub::de…
+      0.0          158,632        153      1,036.8      1,024.0       992       1,216         35.5  void at::native::vectorized_elementwise_kernel<(int)2, at::native::AUnaryFunctor<long, long, long, …
+      0.0          153,253        153      1,001.7        992.0       929       1,216         38.8  void at::native::vectorized_elementwise_kernel<(int)2, at::native::CUDAFunctor_add<long>, std::arra…
+      0.0          138,342        153        904.2        896.0       832       1,120         38.4  void at::native::vectorized_elementwise_kernel<(int)4, at::native::AUnaryFunctor<c10::BFloat16, c10…
+      0.0          120,260        153        786.0        768.0       736         992         37.8  void at::native::vectorized_elementwise_kernel<(int)4, at::native::FillFunctor<float>, std::array<c…
+      0.0          101,381         64      1,584.1      1,568.0     1,472       1,792         80.7  void at::native::elementwise_kernel<(int)128, (int)2, void at::native::gpu_kernel_impl_nocast<at::n…
+      0.0           81,764         64      1,277.6      1,152.0     1,088       2,400        385.8  void at::native::index_elementwise_kernel<(int)128, (int)4, void at::native::gpu_index_kernel<void …
+      0.0           31,745          8      3,968.1      3,936.5     3,712       4,256        190.5  void at_cuda_detail::cub::detail::select::DeviceSelectSweepKernel<at_cuda_detail::cub::detail::sele…
+      0.0           23,169         16      1,448.1      1,424.0     1,280       1,632        118.3  assign_req_to_token_pool                                                                            
+      0.0           21,730         16      1,358.1      1,360.0     1,152       1,696        166.6  get_last_loc_kernel                                                                                 
+      0.0           18,240          8      2,280.0      2,272.0     2,240       2,368         37.3  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortHistogramKernel<at_cuda_detail::cub::d…
+      0.0           16,992          8      2,124.0      2,112.0     2,048       2,272         66.1  void at::native::vectorized_elementwise_kernel<(int)4, at::native::BUnaryFunctor<int, int, int, at:…
+      0.0            9,440          8      1,180.0      1,168.0     1,120       1,248         43.4  void at::native::<unnamed>::CatArrayBatchedCopy_alignedK_contig<at::native::<unnamed>::OpaqueType<(…
+      0.0            8,225          8      1,028.1      1,024.0       992       1,120         43.3  void at_cuda_detail::cub::detail::radix_sort::DeviceRadixSortExclusiveSumKernel<at_cuda_detail::cub…
+      0.0            5,888          8        736.0        736.0       704         768         24.2  void at_cuda_detail::cub::detail::scan::DeviceCompactInitKernel<at_cuda_detail::cub::ScanTileState<…
+
+Processing [/report/profiles/node_5/nsys/d0.sqlite] with [/opt/nvidia/nsight-systems-cli/2026.4.1/target-linux-x64/reports/nvtx_sum.py]... 
+
+ ** NVTX Range Summary (nvtx_sum):
+
+ Time (%)  Total Time (ns)  Instances    Avg (ns)     Med (ns)    Min (ns)    Max (ns)    StdDev (ns)    Style                   Range                
+ --------  ---------------  ---------  ------------  -----------  ---------  -----------  ------------  --------  ------------------------------------
+     66.4   14,731,201,668     78,873     186,771.2    269,479.0        484  106,046,379     779,117.6  StartEnd  NCCL:AllReduce                      
+     13.0    2,890,729,983        160  18,067,062.4  5,237,441.5  4,817,856  304,143,864  54,432,245.2  PushPop   :dflash_target_verify               
+     10.0    2,207,342,791      1,440   1,532,876.9     88,978.5        759   10,030,305   2,577,710.2  StartEnd  NCCL:AllGather                      
+      6.7    1,481,928,797        160   9,262,055.0  2,657,347.0  2,085,029  192,311,281  27,973,693.1  PushPop   :dflash_draft_prepare_forward_sample
+      1.0      231,076,159        160   1,444,226.0  1,231,009.0    908,153    3,671,689     490,453.7  PushPop   :dflash_materialize_draft_kv        
+      0.9      206,649,966     32,480       6,362.4      5,120.0      1,418    1,267,086      19,524.2  StartEnd  NCCL:GroupRuntime                   
+      0.8      173,590,142         16  10,849,383.9     85,687.5     51,325  172,281,867  43,048,669.5  StartEnd  NCCL:CommInit Collection            
+      0.4       83,715,348        160     523,220.9    456,385.0    324,574    1,188,730     174,248.6  PushPop   :dflash_replayssm_commit            
+      0.3       68,016,805        160     425,105.0    369,313.0    298,881    1,001,480     136,790.6  PushPop   :dflash_accept                      
+      0.2       35,803,080        800      44,753.8     39,169.5     23,489      166,215      18,141.6  PushPop   CCCL:cub::DeviceScan::InclusiveScan 
+      0.1       23,106,075        160     144,413.0    115,591.5     89,611      401,881      68,365.7  StartEnd  NCCL:API Group                      
+      0.1       19,587,358        160     122,421.0     99,108.5     72,056      351,450      57,966.4  StartEnd  NCCL:GroupLaunch                    
+      0.1       18,657,059          8   2,332,132.4  2,351,962.0  2,105,165    2,502,099     134,544.4  PushPop   CCCL:cub::DeviceRadixSort           
+      0.0        5,960,181        160      37,251.1     27,998.5     19,843      123,127      20,804.4  StartEnd  NCCL:KernelLaunch                   
+      0.0          530,446          8      66,305.8     62,777.0     41,062      119,196      24,108.4  PushPop   CCCL:cub::DeviceSelect::Unique      
+
+Processing [/report/profiles/node_5/nsys/d0.sqlite] with [/opt/nvidia/nsight-systems-cli/2026.4.1/target-linux-x64/reports/nvtx_gpu_proj_sum.py]... 
+
+ ** NVTX GPU Projection Summary (nvtx_gpu_proj_sum):
+
+                Range                   Style    Total Proj Time (ns)  Total Range Time (ns)  Range Instances  Proj Avg (ns)  Proj Med (ns)  Proj Min (ns)  Proj Max (ns)  Proj StdDev (ns)  Total GPU Ops  Avg GPU Ops  Avg Range Lvl  Avg Num Child
+ ------------------------------------  --------  --------------------  ---------------------  ---------------  -------------  -------------  -------------  -------------  ----------------  -------------  -----------  -------------  -------------
+ :dflash_target_verify                 PushPop         28,286,105,196          2,852,637,482              153  184,876,504.5  183,519,609.0    180,949,744    192,032,157       3,296,693.3            459          3.0            0.0            0.0
+ :dflash_draft_prepare_forward_sample  PushPop          1,310,186,765          1,481,928,797              160    8,188,667.3    6,112,744.5          4,960    240,344,638      18,843,179.5          3,992         24.9            0.0            0.0
+ NCCL:API Group                        StartEnd           128,169,619             22,367,893              153      837,709.9      328,905.0        216,455     12,461,880       2,182,886.6            153          1.0            0.0            1.0
+ NCCL:GroupLaunch                      StartEnd           128,169,619             18,945,082              153      837,709.9      328,905.0        216,455     12,461,880       2,182,886.6            153          1.0            1.0            1.0
+ NCCL:KernelLaunch                     StartEnd           128,169,619              5,795,224              153      837,709.9      328,905.0        216,455     12,461,880       2,182,886.6            153          1.0            2.0            0.0
+ :dflash_materialize_draft_kv          PushPop             87,608,199            222,205,495              153      572,602.6      572,305.0        567,124        588,626           2,418.4          1,836         12.0            0.0            0.0
+ :dflash_replayssm_commit              PushPop              4,910,863             80,548,047              153       32,097.1       32,449.0         28,641         35,681             992.8            765          5.0            0.0            0.0
+ CCCL:cub::DeviceScan::InclusiveScan   PushPop              2,630,000             34,473,275              765        3,437.9        3,104.0          2,976         79,875           3,163.9          1,530          2.0            0.0            0.0
+ :dflash_accept                        PushPop              2,463,661             65,354,814              153       16,102.4       15,872.0         13,889         24,289           1,682.7            459          3.0            0.0            0.0
+ CCCL:cub::DeviceRadixSort             PushPop                521,809             18,657,059                8       65,226.1       65,970.0         56,674         68,130           3,628.4             96         12.0            0.0            0.0
+ CCCL:cub::DeviceSelect::Unique        PushPop                 48,002                530,446                8        6,000.3        5,952.5          5,760          6,304             191.1             16          2.0            0.0            0.0
+
+Processing [/report/profiles/node_5/nsys/d0.sqlite] with [/opt/nvidia/nsight-systems-cli/2026.4.1/target-linux-x64/reports/cuda_api_sum.py]... 
+
+ ** CUDA API Summary (cuda_api_sum):
+
+ Time (%)  Total Time (ns)  Num Calls    Avg (ns)       Med (ns)     Min (ns)   Max (ns)    StdDev (ns)                  Name                
+ --------  ---------------  ---------  -------------  -------------  --------  -----------  ------------  -----------------------------------
+     82.9   24,339,915,697        159  153,081,230.8  168,293,622.0     9,971  233,933,563  50,404,828.9  cudaEventSynchronize               
+      9.8    2,886,748,801        320    9,021,090.0    2,969,324.5   232,892  303,625,849  39,358,940.3  cudaGraphLaunch                    
+      5.1    1,500,800,764        424    3,539,624.4        9,488.0     4,929  187,136,136  25,465,360.8  cudaStreamSynchronize              
+      1.0      290,912,342        932      312,137.7        1,066.0       337  145,452,863   6,694,242.3  cudaThreadExchangeStreamCaptureMode
+      0.4      124,266,257      5,816       21,366.3       13,003.0     5,665    2,165,877      73,994.7  cudaLaunchKernel                   
+      0.3       82,200,378      2,944       27,921.3       20,489.5     7,697    2,053,090      49,464.5  cudaMemcpyAsync                    
+      0.1       42,849,315      2,832       15,130.4       11,695.5     6,114      103,714      10,281.8  cuLaunchKernelEx                   
+      0.1       28,395,935      6,804        4,173.4        2,674.5       750       76,514       4,809.7  cudaEventQuery                     
+      0.0        9,264,477      1,646        5,628.5        4,954.0       846       53,181       4,468.1  cudaEventRecordWithFlags           
+      0.0        7,849,046        480       16,352.2       14,336.0     7,190       72,283       8,410.7  cudaLaunchKernelExC                
+      0.0        5,597,154        208       26,909.4       25,857.0     3,981       84,131      14,636.8  cudaMemsetAsync                    
+      0.0        5,527,685          6      921,280.8      892,572.0   841,781    1,074,944      91,114.7  cudaSetDevice                      
+      0.0        5,310,946        968        5,486.5        5,159.0     1,144       39,237       3,450.0  cudaStreamWaitEvent                
+      0.0        5,277,551      3,702        1,425.6        1,085.5       362       42,406       1,933.2  cudaStreamIsCapturing              
+      0.0        4,780,979      5,976          800.0          480.5       157       35,068       1,321.9  cuKernelGetName                    
+      0.0        4,716,514        320       14,739.1       12,800.0     7,430       50,752       6,496.7  cuLaunchKernel                     
+      0.0        3,089,232        648        4,767.3        4,055.0     1,976       28,911       2,562.9  cudaEventCreateWithFlags           
+      0.0        2,750,711        656        4,193.2        2,432.5     1,088       57,955       4,512.4  cudaEventRecord                    
+      0.0        1,829,159        647        2,827.1        2,506.0     1,571       15,604       1,128.0  cudaEventDestroy                   
+      0.0        1,067,741        160        6,673.4        4,764.5     3,627       41,102       5,688.6  cudaGetFuncBySymbol                
+      0.0        1,007,825        320        3,149.5        2,469.5     1,115       25,034       3,341.4  cuKernelGetFunction                
+      0.0          607,949        320        1,899.8        1,874.5       356       34,920       2,383.3  cudaStreamGetCaptureInfo           
+      0.0          177,089         24        7,378.7        7,460.5       880       30,146       6,206.9  cuLibraryGetKernel                 
+      0.0           90,950          1       90,950.0       90,950.0    90,950       90,950           0.0  cuProfilerStart                    
+      0.0           85,778          6       14,296.3       13,874.0    12,396       16,997       1,630.7  cuDevicePrimaryCtxGetState         
+      0.0            5,827          6          971.2          933.5       614        1,420         287.6  cudaGetLastError                   
+

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/summary.json

@@ -0,0 +1,550 @@
+{
+  "run_id": "kimi3-pd-dflash-nsys-donly-20260902-100251",
+  "benchmarks": {
+    "dflash_speedbench16k_c8_o512.jsonl": {
+      "duration": 118.94434059690684,
+      "completed": 8,
+      "total_input_tokens": 138928,
+      "total_output_tokens": 4096,
+      "input_throughput": 1168.0084929035525,
+      "output_throughput": 34.43627481093049,
+      "mean_ttft_ms": 54464.31915287394,
+      "median_ttft_ms": 59386.413166997954,
+      "p95_ttft_ms": 59393.6821121024,
+      "mean_tpot_ms": 99.104094003175,
+      "median_tpot_ms": 97.14997594733221,
+      "p95_tpot_ms": 134.7547983315567,
+      "mean_e2e_latency_ms": 105106.51118849637,
+      "p95_e2e_latency_ms": 118327.89741209708,
+      "accept_length": 3.8742911153119093
+    }
+  },
+  "request_stages": {
+    "prefill": {
+      "bootstrap_duration": {
+        "count": 104,
+        "mean_s": 5.706239423076923,
+        "p50_s": 4.39433,
+        "p95_s": 11.430574999999997
+      },
+      "queue_duration": {
+        "count": 104,
+        "mean_s": 2.7911374038461543,
+        "p50_s": 0.86754,
+        "p95_s": 8.1849425
+      },
+      "forward_duration": {
+        "count": 104,
+        "mean_s": 18.706726346153847,
+        "p50_s": 22.13745,
+        "p95_s": 28.518517499999998
+      },
+      "entry_time": {
+        "count": 104,
+        "mean_s": 1788315293.2082114,
+        "p50_s": 1788315439.3445,
+        "p95_s": 1788315441.9554498
+      },
+      "transfer_speed": {
+        "count": 104,
+        "mean_s": 0.0055769230769230765,
+        "p50_s": 0.0,
+        "p95_s": 0.02
+      },
+      "transfer_total": {
+        "count": 104,
+        "mean_s": 58.37442307692308,
+        "p50_s": 66.5,
+        "p95_s": 178.25349999999997
+      }
+    },
+    "decode": {
+      "bootstrap_duration": {
+        "count": 13,
+        "mean_s": 0.017917692307692308,
+        "p50_s": 0.0066,
+        "p95_s": 0.06516999999999981
+      },
+      "alloc_wait_duration": {
+        "count": 13,
+        "mean_s": 0.010064615384615384,
+        "p50_s": 0.00438,
+        "p95_s": 0.03411199999999992
+      },
+      "transfer_duration": {
+        "count": 13,
+        "mean_s": 37.433528461538465,
+        "p50_s": 58.99965,
+        "p95_s": 59.095074000000004
+      },
+      "queue_duration": {
+        "count": 13,
+        "mean_s": 0.002277692307692308,
+        "p50_s": 0.0020800000000000003,
+        "p95_s": 0.004147999999999997
+      },
+      "forward_duration": {
+        "count": 13,
+        "mean_s": 31.820733846153846,
+        "p50_s": 38.80487,
+        "p95_s": 65.28663799999998
+      },
+      "entry_time": {
+        "count": 13,
+        "mean_s": 1788315404.1179233,
+        "p50_s": 1788315439.301,
+        "p95_s": 1788315439.3964
+      }
+    }
+  },
+  "nodes": {
+    "node_1": {
+      "samples": 247,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.19671121537255237,
+            "p95": 2.448806256734602,
+            "max": 4.9894216573005234
+          },
+          "rx_gbps": {
+            "mean": 0.00020874026584191777,
+            "p95": 0.0024231344920655528,
+            "max": 0.005370238465032435
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.1976387274092119,
+            "p95": 2.448801477440038,
+            "max": 5.014786380101469
+          },
+          "rx_gbps": {
+            "mean": 0.00020798761430640197,
+            "p95": 0.0024132476278393338,
+            "max": 0.005360326323396621
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 0.43703111726736005,
+            "p95": 5.149259529422662,
+            "max": 10.175180487275092
+          },
+          "rx_gbps": {
+            "mean": 0.0006916967322680225,
+            "p95": 0.007554512417898624,
+            "max": 0.016318888926284577
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.4370123494492523,
+            "p95": 5.149313150520918,
+            "max": 10.051405030397364
+          },
+          "rx_gbps": {
+            "mean": 0.0007060968743812844,
+            "p95": 0.007655775700712061,
+            "max": 0.016172859959430644
+          }
+        }
+      },
+      "gpu_util_pct_mean": 6.684959349593496,
+      "gpu_util_pct_p95": 98.0,
+      "gpu_power_w_mean": 97.43569613821138
+    },
+    "node_2": {
+      "samples": 247,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.2810318380047612,
+            "p95": 3.2371119171413953,
+            "max": 7.360950133157944
+          },
+          "rx_gbps": {
+            "mean": 0.14689441957896468,
+            "p95": 1.8962709331143928,
+            "max": 3.7951525576148755
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.3006056845498361,
+            "p95": 3.6767661168143713,
+            "max": 7.361287067990863
+          },
+          "rx_gbps": {
+            "mean": 0.1468937723052323,
+            "p95": 1.8960286973636142,
+            "max": 3.7960536511858822
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 0.511268927317164,
+            "p95": 5.930135064722518,
+            "max": 10.24509823942087
+          },
+          "rx_gbps": {
+            "mean": 0.0008662802681295936,
+            "p95": 0.009418630444433545,
+            "max": 0.016417783283060074
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.5112357785827406,
+            "p95": 5.930085752260073,
+            "max": 10.24569261118168
+          },
+          "rx_gbps": {
+            "mean": 0.0008665952974817817,
+            "p95": 0.009327597256139034,
+            "max": 0.01660210110858771
+          }
+        }
+      },
+      "gpu_util_pct_mean": 6.8602642276422765,
+      "gpu_util_pct_p95": 97.0,
+      "gpu_power_w_mean": 94.60943597560976
+    },
+    "node_3": {
+      "samples": 247,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.28244125942221077,
+            "p95": 3.474863932684786,
+            "max": 7.390116374861817
+          },
+          "rx_gbps": {
+            "mean": 0.22017332653438873,
+            "p95": 2.821379507835419,
+            "max": 5.727497709441206
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.30148454151572035,
+            "p95": 3.678250234489492,
+            "max": 7.7171280283518975
+          },
+          "rx_gbps": {
+            "mean": 0.22015928735917567,
+            "p95": 2.8213087637015772,
+            "max": 5.727480279270306
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 0.594083356519429,
+            "p95": 6.809866312237727,
+            "max": 11.094272531217168
+          },
+          "rx_gbps": {
+            "mean": 0.001059726284602477,
+            "p95": 0.011533708909761953,
+            "max": 0.02109696714543949
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.5750177862190479,
+            "p95": 6.4983839009316515,
+            "max": 11.093663114874984
+          },
+          "rx_gbps": {
+            "mean": 0.0010482726754870184,
+            "p95": 0.011550378772163632,
+            "max": 0.02126643957772755
+          }
+        }
+      },
+      "gpu_util_pct_mean": 6.145833333333333,
+      "gpu_util_pct_p95": 95.0,
+      "gpu_power_w_mean": 94.10256605691058
+    },
+    "node_4": {
+      "samples": 247,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.3016460922133353,
+            "p95": 3.6734823779655197,
+            "max": 11.290051526154093
+          },
+          "rx_gbps": {
+            "mean": 0.2934530126714333,
+            "p95": 3.802668043603367,
+            "max": 7.422183990824773
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.2826127263454641,
+            "p95": 3.581947883167566,
+            "max": 8.333866158115198
+          },
+          "rx_gbps": {
+            "mean": 0.29341069050467655,
+            "p95": 3.802537764004363,
+            "max": 7.421673307823631
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 0.7038972190288018,
+            "p95": 5.23357462770881,
+            "max": 5.795327669403328
+          },
+          "rx_gbps": {
+            "mean": 0.012939838938355359,
+            "p95": 0.09258813072702647,
+            "max": 0.10143637305473124
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.7038952389146154,
+            "p95": 5.255231997187769,
+            "max": 5.638958240697699
+          },
+          "rx_gbps": {
+            "mean": 0.012932577217685239,
+            "p95": 0.09173061535260435,
+            "max": 0.1008901983024635
+          }
+        }
+      },
+      "gpu_util_pct_mean": 4.926829268292683,
+      "gpu_util_pct_p95": 50.24999999999932,
+      "gpu_power_w_mean": 93.85190040650407
+    },
+    "node_5": {
+      "samples": 244,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.09147217980663752,
+            "p95": 0.39076875563199126,
+            "max": 0.41354140444490634
+          },
+          "rx_gbps": {
+            "mean": 8.571175505238827,
+            "p95": 39.10953668212727,
+            "max": 39.68521758158616
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.0018967064943002283,
+            "p95": 0.014886225611767119,
+            "max": 0.021374039400711815
+          },
+          "rx_gbps": {
+            "mean": 0.3349493177503921,
+            "p95": 2.7266742070845775,
+            "max": 3.9465511175643933
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 8.238567631617629,
+            "p95": 39.11013514578054,
+            "max": 39.68045966645756
+          },
+          "rx_gbps": {
+            "mean": 0.425256262029413,
+            "p95": 2.761117676387736,
+            "max": 4.44544748666892
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.001907776973833361,
+            "p95": 0.015328351736200549,
+            "max": 0.025009791132251543
+          },
+          "rx_gbps": {
+            "mean": 0.33494895892107396,
+            "p95": 2.737567318921787,
+            "max": 4.443166692018081
+          }
+        }
+      },
+      "gpu_util_pct_mean": 40.016460905349795,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 147.0264146090535
+    },
+    "node_6": {
+      "samples": 247,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.09090084192441149,
+            "p95": 0.392453428271813,
+            "max": 0.41753512896939926
+          },
+          "rx_gbps": {
+            "mean": 8.467189663210501,
+            "p95": 39.13998568870832,
+            "max": 39.6805917451515
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.001848038526394207,
+            "p95": 0.012435236007340122,
+            "max": 0.02352170037979479
+          },
+          "rx_gbps": {
+            "mean": 0.33093581634741365,
+            "p95": 2.8217868206864307,
+            "max": 3.413060363540618
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 8.137505882243303,
+            "p95": 39.14612557327379,
+            "max": 39.620431657618965
+          },
+          "rx_gbps": {
+            "mean": 0.4195010919616931,
+            "p95": 2.7645839434081516,
+            "max": 3.9444492438369974
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.0018573931175485212,
+            "p95": 0.01313552715284922,
+            "max": 0.023505101084878393
+          },
+          "rx_gbps": {
+            "mean": 0.3309191665927233,
+            "p95": 2.736635026954175,
+            "max": 3.9435649803194583
+          }
+        }
+      },
+      "gpu_util_pct_mean": 42.676321138211385,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 96.91137195121951
+    },
+    "node_7": {
+      "samples": 248,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.09000891134949884,
+            "p95": 0.390825878072398,
+            "max": 0.4136010849700234
+          },
+          "rx_gbps": {
+            "mean": 8.432425618379876,
+            "p95": 39.13791853744553,
+            "max": 39.62655275599643
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.001864230626408724,
+            "p95": 0.013890404056949208,
+            "max": 0.02139894136459566
+          },
+          "rx_gbps": {
+            "mean": 0.3295252995646731,
+            "p95": 2.7810589500284086,
+            "max": 4.3985627292469145
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 8.10446846731425,
+            "p95": 39.13686085865652,
+            "max": 39.541044441165766
+          },
+          "rx_gbps": {
+            "mean": 0.4176783491867553,
+            "p95": 2.6963162856942113,
+            "max": 4.236679856881758
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.0018435741167394902,
+            "p95": 0.014607594762173794,
+            "max": 0.02004328508762458
+          },
+          "rx_gbps": {
+            "mean": 0.3295813504869624,
+            "p95": 2.628810010862299,
+            "max": 4.234286815997679
+          }
+        }
+      },
+      "gpu_util_pct_mean": 42.50657894736842,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 96.73924089068825
+    },
+    "node_8": {
+      "samples": 248,
+      "hcas": {
+        "mlx5_0": {
+          "tx_gbps": {
+            "mean": 0.08989748693510208,
+            "p95": 0.38864787736552814,
+            "max": 0.4140660785419288
+          },
+          "rx_gbps": {
+            "mean": 8.432211157058987,
+            "p95": 39.112599954595694,
+            "max": 39.65849656360841
+          }
+        },
+        "mlx5_1": {
+          "tx_gbps": {
+            "mean": 0.0018543402958808187,
+            "p95": 0.014757171451893869,
+            "max": 0.02497321575731713
+          },
+          "rx_gbps": {
+            "mean": 0.32965106424752544,
+            "p95": 2.5953188927889803,
+            "max": 3.944181414041341
+          }
+        },
+        "mlx5_2": {
+          "tx_gbps": {
+            "mean": 8.104612142794483,
+            "p95": 39.12364291569094,
+            "max": 39.58082795816169
+          },
+          "rx_gbps": {
+            "mean": 0.4176858333912433,
+            "p95": 2.7612561251348278,
+            "max": 3.508535811256942
+          }
+        },
+        "mlx5_3": {
+          "tx_gbps": {
+            "mean": 0.0018625721560511367,
+            "p95": 0.014376258008787876,
+            "max": 0.022403425198762445
+          },
+          "rx_gbps": {
+            "mean": 0.32965378319071004,
+            "p95": 2.740893789748552,
+            "max": 3.402528380671608
+          }
+        }
+      },
+      "gpu_util_pct_mean": 42.43927125506073,
+      "gpu_util_pct_p95": 100.0,
+      "gpu_power_w_mean": 95.8680971659919
+    }
+  }
+}

experiments/pro6000/kimi3_pro6000_pd_dflash_attribution/results/kimi3-pd-dflash-nsys-donly-20260902-100251/telemetry/node_1.jsonl

@@ -0,0 +1,247 @@
+{"wall_time":1788315330.9709303,"monotonic":2406205.967260346,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:31.119","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.91,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:31.133","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.42,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:31.144","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.78,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:31.155","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:31.165","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:31.175","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.12,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:31.185","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.72,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:31.194","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.74,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315331.9717178,"monotonic":2406206.968045663,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:32.120","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.59,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:32.137","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.46,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:32.150","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:32.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:32.173","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:32.184","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.16,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:32.194","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.2,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:32.205","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.78,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315332.9724934,"monotonic":2406207.968820822,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:33.130","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:33.144","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:33.155","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:33.166","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.67,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:33.176","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:33.187","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:33.196","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:33.205","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.5,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315333.9732683,"monotonic":2406208.969596084,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:34.125","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:34.139","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:34.150","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:34.160","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:34.170","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:34.179","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:34.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:34.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315334.97406,"monotonic":2406209.970387658,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:35.115","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:35.129","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:35.140","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:35.151","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:35.161","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:35.170","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:35.179","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:35.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315335.9748485,"monotonic":2406210.971177019,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:36.126","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:36.141","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:36.152","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:36.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:36.173","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:36.182","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:36.191","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:36.200","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315336.9756355,"monotonic":2406211.971962638,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:37.126","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:37.142","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:37.153","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:37.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:37.173","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:37.183","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:37.192","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:37.201","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315337.9757457,"monotonic":2406212.972071287,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:38.125","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:38.141","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:38.152","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:38.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:38.173","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:38.186","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:38.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:38.209","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315338.9765427,"monotonic":2406213.972868366,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:39.154","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:39.173","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:39.188","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:39.201","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:39.214","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:39.228","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:39.242","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:39.252","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315339.9772978,"monotonic":2406214.97362173,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:40.124","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:40.140","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:40.151","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:40.162","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:40.172","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:40.184","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:40.193","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:40.202","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315340.9777298,"monotonic":2406215.974055022,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:41.126","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:41.147","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:41.169","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:41.191","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:41.214","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:41.230","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:41.241","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:41.251","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315341.9784784,"monotonic":2406216.974805442,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:42.140","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:42.156","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:42.167","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:42.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:42.190","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:42.200","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:42.210","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:42.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315342.9792387,"monotonic":2406217.975565581,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:43.126","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:43.139","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:43.150","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:43.160","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:43.170","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:43.179","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:43.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:43.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315343.979735,"monotonic":2406218.976062237,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:44.130","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:44.144","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:44.155","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:44.165","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:44.175","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:44.184","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:44.193","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:44.201","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315344.9805272,"monotonic":2406219.976854488,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:45.121","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:45.137","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:45.148","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:45.158","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:45.168","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:45.178","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:45.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:45.198","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315345.9810438,"monotonic":2406220.977368402,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:46.133","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:46.148","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:46.159","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:46.170","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:46.180","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.83,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:46.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:46.198","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:46.208","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315346.981848,"monotonic":2406221.978173544,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:47.140","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":136.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:47.156","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:47.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:47.180","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:47.191","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:47.201","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:47.211","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:47.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315347.9826522,"monotonic":2406222.978976482,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:48.117","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:48.132","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:48.144","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:48.155","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:48.167","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:48.178","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:48.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:48.200","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315348.9834664,"monotonic":2406223.979791473,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:49.129","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:49.146","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:49.158","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:49.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:49.179","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:49.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:49.198","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:49.208","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315349.983751,"monotonic":2406224.980076235,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:50.130","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:50.146","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:50.157","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:50.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:50.177","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:50.187","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:50.196","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:50.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315350.9845588,"monotonic":2406225.98088403,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:51.129","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":134.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:51.145","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:51.157","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:51.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:51.180","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:51.190","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:51.200","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:51.211","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315351.985367,"monotonic":2406226.981692598,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:52.136","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:52.150","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:52.161","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:52.172","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:52.182","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:52.191","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:52.200","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:52.209","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315352.9861524,"monotonic":2406227.982479565,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:53.141","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:53.157","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:53.169","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:53.179","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:53.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:53.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:53.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:53.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315353.9869442,"monotonic":2406228.983271441,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:54.135","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:54.149","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:54.160","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:54.170","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:54.180","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:54.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:54.198","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:54.207","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315354.987718,"monotonic":2406229.984045559,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:55.127","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:55.141","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:55.152","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:55.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:55.173","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:55.182","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:55.192","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:55.200","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315355.988154,"monotonic":2406230.984481424,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:56.137","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:56.153","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:56.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:56.174","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:56.183","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:56.192","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:56.201","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:56.210","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315356.98827,"monotonic":2406231.984595656,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:57.144","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:57.160","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:57.171","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:57.181","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:57.195","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.27,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:57.208","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:57.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:57.233","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315357.9884334,"monotonic":2406232.98476039,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:58.135","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:58.148","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:58.159","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:58.170","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:58.180","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:58.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:58.198","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:58.208","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315358.9892223,"monotonic":2406233.985549688,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:15:59.138","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:15:59.150","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:15:59.160","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:15:59.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:15:59.179","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:15:59.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:15:59.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:15:59.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315359.9897885,"monotonic":2406234.98611597,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:00.138","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":134.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:00.156","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:00.166","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:00.177","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:00.187","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:00.196","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:00.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:00.214","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315360.9905617,"monotonic":2406235.986887025,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:01.142","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":133.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:01.157","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:01.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:01.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:01.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:01.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:01.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:01.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315361.9913473,"monotonic":2406236.987674511,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:02.139","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:02.155","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:02.167","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:02.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:02.189","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:02.199","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:02.209","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:02.218","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315362.992165,"monotonic":2406237.988490287,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:03.146","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.21,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:03.162","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:03.173","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:03.184","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:03.195","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:03.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:03.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:03.225","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315363.992967,"monotonic":2406238.989292136,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:04.143","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":134.92,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:04.158","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:04.169","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:04.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:04.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:04.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:04.207","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:04.216","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315364.9937592,"monotonic":2406239.990086243,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:05.138","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:05.154","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:05.165","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:05.175","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:05.185","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:05.194","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:05.203","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:05.212","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315365.994543,"monotonic":2406240.990868714,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:06.145","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:06.160","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:06.171","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:06.182","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:06.192","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:06.202","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:06.211","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:06.221","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315366.9953516,"monotonic":2406241.99167635,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:07.137","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:07.152","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:07.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:07.173","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:07.182","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.51,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:07.191","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:07.200","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:07.209","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315367.9957607,"monotonic":2406242.992086022,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:08.133","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:08.148","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:08.158","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:08.169","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:08.179","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:08.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:08.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.81,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:08.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315368.9965818,"monotonic":2406243.992907263,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:09.151","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:09.170","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:09.182","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:09.194","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:09.205","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:09.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:09.225","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:09.234","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315369.9973998,"monotonic":2406244.993723752,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:10.149","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:10.163","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:10.174","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:10.187","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:10.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:10.207","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:10.217","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:10.226","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315370.9982004,"monotonic":2406245.9945264,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:11.144","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:11.162","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:11.174","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:11.185","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:11.196","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:11.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:11.216","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:11.225","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315371.9989967,"monotonic":2406246.995322866,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:12.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:12.182","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:12.193","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:12.203","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:12.213","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:12.222","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:12.231","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:12.240","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315372.9997303,"monotonic":2406247.996056509,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:13.144","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:13.162","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:13.173","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:13.183","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:13.196","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:13.208","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:13.218","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:13.227","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315374.000531,"monotonic":2406248.996856657,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:14.152","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:14.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:14.203","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:14.226","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:14.247","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:14.258","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:14.269","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:14.279","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315375.0012777,"monotonic":2406249.997602871,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:15.151","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:15.166","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:15.177","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:15.187","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:15.199","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:15.209","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:15.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:15.231","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315376.0020704,"monotonic":2406250.998396887,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:16.141","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:16.156","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:16.167","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:16.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:16.187","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:16.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:16.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:16.216","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315377.0028853,"monotonic":2406251.999211078,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:17.141","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:17.156","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:17.167","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:17.177","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:17.187","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:17.196","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:17.205","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:17.214","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315378.0037305,"monotonic":2406253.000055845,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:18.141","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:18.157","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:18.168","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:18.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:18.188","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:18.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:18.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:18.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315379.0045455,"monotonic":2406254.000870619,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:19.150","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:19.165","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:19.176","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:19.187","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:19.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:19.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:19.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:19.224","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315380.0053346,"monotonic":2406255.00166149,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:20.151","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:20.165","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:20.176","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:20.187","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:20.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:20.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:20.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:20.224","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315381.0061245,"monotonic":2406256.002452233,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:21.150","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:21.164","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.22,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:21.178","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:21.191","gpu_util_pct":0.0,"memory_used_mib":52569.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:21.202","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:21.212","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:21.221","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:21.231","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315382.0069168,"monotonic":2406257.003244339,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:22.144","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":133.42,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:22.160","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":100.94,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:22.172","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":109.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:22.182","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":116.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:22.192","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:22.202","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:22.211","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:22.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315383.0077224,"monotonic":2406258.004049354,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:23.153","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":109.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:23.168","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":116.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:23.179","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":101.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:23.190","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":108.31,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:23.201","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:23.210","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:23.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:23.229","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315384.0085185,"monotonic":2406259.004845846,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:24.145","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":80.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:24.159","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.6,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:24.171","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:24.181","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:24.192","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:24.201","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:24.211","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:24.222","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315385.009315,"monotonic":2406260.005642065,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:25.137","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:25.152","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:25.163","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:25.174","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:25.184","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:25.193","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:25.202","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:25.211","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315386.0101235,"monotonic":2406261.006450784,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:26.149","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":80.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:26.164","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:26.175","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:26.185","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:26.194","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:26.204","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:26.213","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:26.222","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315387.0109236,"monotonic":2406262.007250966,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:27.150","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":80.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:27.164","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:27.176","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:27.186","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:27.196","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:27.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:27.215","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:27.224","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315388.0117319,"monotonic":2406263.008059346,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:28.152","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":80.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:28.167","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:28.177","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:28.187","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:28.197","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:28.206","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:28.214","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:28.223","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315389.0125406,"monotonic":2406264.008867754,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:29.148","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:29.162","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:29.173","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:29.184","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:29.193","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:29.203","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:29.212","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:29.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315390.0126705,"monotonic":2406265.008998031,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:30.097","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:30.112","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:30.122","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:30.133","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:30.142","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:30.152","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:30.161","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:30.170","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315391.013522,"monotonic":2406266.009849468,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:31.157","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":80.22,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:31.170","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:31.181","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:31.192","gpu_util_pct":0.0,"memory_used_mib":57021.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:31.202","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:31.211","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:31.220","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:31.229","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":82.36,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315392.0143127,"monotonic":2406267.01064008,"hcas":{"mlx5_0":{"port_xmit_data":138737916718344,"port_rcv_data":138763972754171,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837124529747,"port_rcv_data":1192118453057,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138856914082405,"port_rcv_data":139149759536598,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2186949558583,"port_rcv_data":1505258660532,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:32.156","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":139.8,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:32.172","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":117.71,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:32.184","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":131.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:32.195","gpu_util_pct":100.0,"memory_used_mib":57021.0,"power_w":138.83,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:32.207","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:32.217","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:32.225","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":77.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:32.234","gpu_util_pct":0.0,"memory_used_mib":57413.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315393.0151134,"monotonic":2406268.011439192,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:33.163","gpu_util_pct":80.0,"memory_used_mib":59345.0,"power_w":270.27,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:33.180","gpu_util_pct":2.0,"memory_used_mib":59345.0,"power_w":284.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:33.192","gpu_util_pct":59.0,"memory_used_mib":59345.0,"power_w":305.61,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:33.203","gpu_util_pct":35.0,"memory_used_mib":59345.0,"power_w":278.38,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:33.214","gpu_util_pct":62.0,"memory_used_mib":64333.0,"power_w":166.34,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:33.231","gpu_util_pct":99.0,"memory_used_mib":64333.0,"power_w":256.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:33.245","gpu_util_pct":99.0,"memory_used_mib":64333.0,"power_w":178.67,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:33.258","gpu_util_pct":99.0,"memory_used_mib":64333.0,"power_w":284.47,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315394.0157154,"monotonic":2406269.012042147,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:34.158","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":106.81,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:34.171","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":81.71,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:34.182","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":109.3,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:34.193","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":89.71,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:34.202","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":231.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:34.212","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":245.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:34.222","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":237.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:34.231","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":242.05,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315395.0165095,"monotonic":2406270.012836857,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:35.163","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:35.179","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:35.192","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:35.205","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:35.216","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":96.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:35.227","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.09,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:35.238","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":105.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:35.248","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":103.67,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315396.017281,"monotonic":2406271.013608625,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:36.165","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:36.179","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:36.190","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:36.200","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:36.210","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:36.220","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.92,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:36.229","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:36.238","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315397.0180738,"monotonic":2406272.014401059,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:37.172","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:37.187","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:37.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:37.211","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:37.223","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:37.234","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:37.246","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:37.257","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315398.0188344,"monotonic":2406273.015161596,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:38.166","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:38.180","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:38.191","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:38.201","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:38.210","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:38.219","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:38.228","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:38.237","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315399.0196283,"monotonic":2406274.015955732,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:39.164","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:39.178","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:39.189","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:39.200","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:39.210","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:39.220","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:39.229","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:39.238","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315400.0197191,"monotonic":2406275.016046712,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:40.170","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:40.188","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:40.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:40.211","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":81.53,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:40.223","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:40.234","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:40.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:40.256","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315401.0204916,"monotonic":2406276.016818949,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:41.173","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:41.186","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:41.197","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:41.208","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:41.217","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:41.227","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:41.236","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:41.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315402.0212727,"monotonic":2406277.017600326,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:42.174","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:42.188","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:42.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:42.209","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:42.219","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:42.229","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:42.240","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:42.250","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315403.0220454,"monotonic":2406278.018372922,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857088602177,"port_rcv_data":139149759793185,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187124080760,"port_rcv_data":1505258919106,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:43.165","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:43.177","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:43.189","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:43.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:43.209","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:43.219","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:43.228","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:43.237","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315404.0228405,"monotonic":2406279.019168342,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:44.173","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:44.187","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:44.198","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:44.208","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.85,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:44.218","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.01,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:44.228","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":79.45,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:44.237","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:44.246","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":83.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315405.0236294,"monotonic":2406280.019956523,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:45.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:45.212","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:45.223","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:45.234","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:45.243","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.8,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:45.253","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:45.262","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:45.271","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315406.0243893,"monotonic":2406281.020717349,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:46.170","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:46.184","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:46.196","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:46.207","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:46.217","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:46.227","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:46.236","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:46.246","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.43,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315407.0251784,"monotonic":2406282.02150592,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:47.170","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:47.193","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:47.217","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:47.240","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:47.261","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:47.272","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:47.283","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:47.293","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315408.0259185,"monotonic":2406283.022246321,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:48.170","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:48.184","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:48.196","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:48.207","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:48.217","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:48.227","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:48.238","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:48.249","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315409.0267146,"monotonic":2406284.023041854,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:49.177","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:49.191","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:49.203","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:49.214","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:49.224","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:49.234","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:49.243","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:49.252","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315410.0274858,"monotonic":2406285.023813531,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:50.169","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:50.183","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:50.194","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.09,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:50.204","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:50.213","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:50.222","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:50.231","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:50.240","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315411.0277286,"monotonic":2406286.024056296,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:51.170","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:51.184","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:51.195","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:51.205","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:51.215","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:51.224","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:51.233","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:51.242","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315412.0285091,"monotonic":2406287.024837989,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:52.130","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:52.142","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:52.153","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:52.163","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:52.173","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:52.182","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:52.191","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:52.200","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315413.0293868,"monotonic":2406288.025712525,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:53.176","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:53.190","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:53.201","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:53.212","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:53.222","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:53.232","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:53.241","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:53.250","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315414.030175,"monotonic":2406289.026501868,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:54.183","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:54.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:54.210","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:54.221","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:54.231","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:54.241","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:54.250","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:54.259","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315415.0309877,"monotonic":2406290.027313083,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:55.173","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:55.189","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:55.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:55.210","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:55.219","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:55.228","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:55.237","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:55.246","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315416.031736,"monotonic":2406291.028061401,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:56.182","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:56.197","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:56.209","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:56.220","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:56.229","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:56.239","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:56.248","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:56.257","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315417.032539,"monotonic":2406292.028864394,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:57.190","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:57.205","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:57.216","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:57.226","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:57.236","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.65,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:57.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:57.254","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:57.263","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315418.0333164,"monotonic":2406293.029643506,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:58.172","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:58.186","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:58.197","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:58.207","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:58.217","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:58.226","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:58.235","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:58.244","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315419.034128,"monotonic":2406294.03045493,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:16:59.191","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:16:59.205","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:16:59.216","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:16:59.226","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:16:59.236","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.13,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:16:59.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:16:59.254","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:16:59.263","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315420.0344832,"monotonic":2406295.030811037,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:00.178","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:00.192","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:00.203","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:00.213","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:00.223","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:00.233","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:00.242","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:00.251","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315421.0352778,"monotonic":2406296.031604834,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:01.181","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:01.195","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:01.206","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:01.217","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:01.226","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:01.236","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:01.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:01.254","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315422.0357192,"monotonic":2406297.032046979,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:02.187","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:02.202","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:02.213","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:02.224","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:02.234","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:02.243","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:02.252","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:02.261","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315423.036512,"monotonic":2406298.032838909,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:03.189","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.75,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:03.203","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:03.215","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:03.226","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:03.236","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:03.246","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.17,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:03.256","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:03.265","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315424.0370293,"monotonic":2406299.033356183,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:04.191","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:04.203","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:04.212","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:04.220","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:04.229","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:04.237","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:04.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:04.253","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315425.0378265,"monotonic":2406300.034153634,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:05.185","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:05.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:05.210","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:05.220","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:05.230","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:05.239","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:05.249","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:05.257","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315426.0386224,"monotonic":2406301.034949738,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:06.189","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:06.203","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:06.214","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:06.224","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:06.234","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:06.244","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:06.253","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:06.262","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315427.0394168,"monotonic":2406302.035743344,"hcas":{"mlx5_0":{"port_xmit_data":138737993303304,"port_rcv_data":138763972829796,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:07.184","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:07.198","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:07.209","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:07.219","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:07.229","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:07.238","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:07.247","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:07.256","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315428.0402188,"monotonic":2406303.036546286,"hcas":{"mlx5_0":{"port_xmit_data":138737993303634,"port_rcv_data":138763972830074,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837201114707,"port_rcv_data":1192118528496,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857144990063,"port_rcv_data":139149759938916,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187180461394,"port_rcv_data":1505259087260,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:08.185","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.02,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:08.194","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.71,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:08.202","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:08.211","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:08.219","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:08.227","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.34,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:08.234","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:08.242","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315429.0410197,"monotonic":2406304.037348277,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:09.194","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.13,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:09.208","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.29,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:09.219","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:09.230","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:09.241","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:09.250","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:09.260","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.24,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:09.270","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.46,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315430.0418012,"monotonic":2406305.03812875,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:10.197","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.26,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:10.210","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.31,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:10.222","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.44,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:10.234","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.25,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:10.244","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.91,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:10.253","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:10.262","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:10.271","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315431.0425718,"monotonic":2406306.038899559,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:11.193","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:11.207","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.04,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:11.218","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:11.228","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.63,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:11.238","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:11.248","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.48,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:11.257","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:11.266","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315432.0431244,"monotonic":2406307.039451634,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:12.188","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:12.201","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:12.212","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:12.223","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":78.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:12.233","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:12.242","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:12.251","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:12.260","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315433.0437183,"monotonic":2406308.04004594,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:13.189","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:13.203","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.07,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:13.214","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:13.224","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:13.233","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.54,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:13.243","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.04,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:13.251","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:13.260","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315434.0445113,"monotonic":2406309.040838775,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:14.193","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.43,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:14.206","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.04,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:14.217","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.16,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:14.227","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.23,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:14.237","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:14.246","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.01,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:14.255","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.78,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:14.264","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.29,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315435.045302,"monotonic":2406310.04162972,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:15.189","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.09,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:15.202","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:15.213","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:15.224","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:15.234","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:15.243","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:15.252","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:15.261","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.92,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315436.0460954,"monotonic":2406311.042422805,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:16.199","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:16.214","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:16.226","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:16.236","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:16.246","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.92,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:16.257","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:16.266","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:16.275","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315437.0469055,"monotonic":2406312.043231467,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:17.203","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.0,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:17.218","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:17.230","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:17.241","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:17.252","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:17.262","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:17.272","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:17.281","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315438.0477033,"monotonic":2406313.044030619,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:18.201","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.24,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:18.211","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:18.220","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:18.228","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:18.237","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:18.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:18.254","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:18.265","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315439.0484936,"monotonic":2406314.044821391,"hcas":{"mlx5_0":{"port_xmit_data":138738072766394,"port_rcv_data":138763972917440,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837281370743,"port_rcv_data":1192118616551,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:19.173","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:19.185","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:19.196","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:19.206","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:19.216","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:19.225","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:19.235","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:19.245","gpu_util_pct":0.0,"memory_used_mib":64333.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315440.049308,"monotonic":2406315.045635566,"hcas":{"mlx5_0":{"port_xmit_data":138738149351354,"port_rcv_data":138763972992894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837357955703,"port_rcv_data":1192118691835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857260218634,"port_rcv_data":139149760061448,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187295690386,"port_rcv_data":1505259209017,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:20.229","gpu_util_pct":92.0,"memory_used_mib":59345.0,"power_w":167.16,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:20.243","gpu_util_pct":95.0,"memory_used_mib":59345.0,"power_w":266.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:20.257","gpu_util_pct":91.0,"memory_used_mib":59345.0,"power_w":173.87,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:20.270","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":278.68,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:20.281","gpu_util_pct":54.0,"memory_used_mib":64445.0,"power_w":89.26,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:20.293","gpu_util_pct":98.0,"memory_used_mib":64445.0,"power_w":109.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:20.306","gpu_util_pct":97.0,"memory_used_mib":64445.0,"power_w":157.17,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:20.319","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":129.35,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315441.050032,"monotonic":2406316.046359607,"hcas":{"mlx5_0":{"port_xmit_data":138738149351354,"port_rcv_data":138763972992894,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837357955703,"port_rcv_data":1192118691835,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857434771459,"port_rcv_data":139149760350747,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187470255399,"port_rcv_data":1505259510488,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:21.196","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":318.56,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:21.210","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":343.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:21.222","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":343.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:21.233","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":327.24,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:21.242","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":297.21,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:21.253","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":300.38,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:21.262","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":322.04,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:21.272","gpu_util_pct":100.0,"memory_used_mib":64445.0,"power_w":293.47,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315442.0508146,"monotonic":2406317.047141579,"hcas":{"mlx5_0":{"port_xmit_data":138738225936314,"port_rcv_data":138763973069077,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837434540663,"port_rcv_data":1192118767413,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857491134028,"port_rcv_data":139149760469688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187526589103,"port_rcv_data":1505259629526,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:22.194","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":310.42,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:22.229","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":279.08,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:22.262","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":264.73,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:22.291","gpu_util_pct":91.0,"memory_used_mib":59345.0,"power_w":244.87,"sm_clock_mhz":2287.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:22.310","gpu_util_pct":7.0,"memory_used_mib":64557.0,"power_w":290.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:22.322","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":278.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:22.334","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":264.19,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:22.349","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":265.64,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315443.0515153,"monotonic":2406318.04784237,"hcas":{"mlx5_0":{"port_xmit_data":138738302521274,"port_rcv_data":138763973144986,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837511125623,"port_rcv_data":1192118843341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857665683700,"port_rcv_data":139149760755762,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187701137150,"port_rcv_data":1505259913691,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:23.195","gpu_util_pct":3.0,"memory_used_mib":59345.0,"power_w":204.51,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:23.212","gpu_util_pct":86.0,"memory_used_mib":59345.0,"power_w":200.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:23.225","gpu_util_pct":26.0,"memory_used_mib":59345.0,"power_w":206.43,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:23.237","gpu_util_pct":52.0,"memory_used_mib":59345.0,"power_w":197.39,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:23.248","gpu_util_pct":82.0,"memory_used_mib":64557.0,"power_w":264.16,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:23.259","gpu_util_pct":97.0,"memory_used_mib":64557.0,"power_w":321.3,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:23.269","gpu_util_pct":97.0,"memory_used_mib":64557.0,"power_w":293.5,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:23.279","gpu_util_pct":96.0,"memory_used_mib":64557.0,"power_w":299.41,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315444.052313,"monotonic":2406319.048638253,"hcas":{"mlx5_0":{"port_xmit_data":138738379106234,"port_rcv_data":138763973220411,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837587710583,"port_rcv_data":1192118918697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138857869528132,"port_rcv_data":139149761097602,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2187904971501,"port_rcv_data":1505260263506,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:24.202","gpu_util_pct":97.0,"memory_used_mib":59345.0,"power_w":269.73,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:24.219","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":274.64,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:24.231","gpu_util_pct":96.0,"memory_used_mib":59345.0,"power_w":278.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:24.242","gpu_util_pct":98.0,"memory_used_mib":59345.0,"power_w":248.7,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:24.253","gpu_util_pct":97.0,"memory_used_mib":64557.0,"power_w":233.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:24.263","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":248.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:24.274","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":263.46,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:24.284","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":216.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315445.0530934,"monotonic":2406320.049418842,"hcas":{"mlx5_0":{"port_xmit_data":138738455691194,"port_rcv_data":138763973295509,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837664295543,"port_rcv_data":1192118993733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138858044065391,"port_rcv_data":139149761371614,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2188079505054,"port_rcv_data":1505260533300,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:25.194","gpu_util_pct":61.0,"memory_used_mib":59345.0,"power_w":263.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:25.209","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":227.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:25.221","gpu_util_pct":39.0,"memory_used_mib":59345.0,"power_w":239.44,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:25.232","gpu_util_pct":13.0,"memory_used_mib":59345.0,"power_w":228.14,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:25.242","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":265.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:25.254","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":278.6,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:25.265","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":267.48,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:25.276","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":277.03,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315446.0538845,"monotonic":2406321.050211357,"hcas":{"mlx5_0":{"port_xmit_data":138738455691719,"port_rcv_data":138763973295982,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837664295543,"port_rcv_data":1192118993733,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138858218606528,"port_rcv_data":139149761654608,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2188254050382,"port_rcv_data":1505260819043,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:26.196","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":186.74,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:26.214","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":225.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:26.228","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":200.53,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:26.240","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":200.13,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:26.251","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":206.47,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:26.263","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":209.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:26.273","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":243.49,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:26.284","gpu_util_pct":0.0,"memory_used_mib":64557.0,"power_w":220.94,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315447.0546787,"monotonic":2406322.051004159,"hcas":{"mlx5_0":{"port_xmit_data":138738611734759,"port_rcv_data":138763973463935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837821131859,"port_rcv_data":1192119161376,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138858224146079,"port_rcv_data":139149761681272,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2188259580277,"port_rcv_data":1505260847674,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:27.198","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":224.56,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:27.214","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":207.26,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:27.225","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":193.29,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:27.236","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":187.4,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:27.246","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":191.64,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:27.257","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":209.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:27.267","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":192.84,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:27.278","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":222.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315448.0554717,"monotonic":2406323.0517971,"hcas":{"mlx5_0":{"port_xmit_data":138738611740061,"port_rcv_data":138763973465745,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837821136539,"port_rcv_data":1192119162616,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138858513882902,"port_rcv_data":139149762051410,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2188549325336,"port_rcv_data":1505261225935,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:28.182","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":264.93,"sm_clock_mhz":2392.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:28.197","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":279.3,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:28.209","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":302.01,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:28.219","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":292.03,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:28.229","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":254.1,"sm_clock_mhz":2280.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:28.239","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":270.11,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:28.248","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":257.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:28.258","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":246.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315449.055752,"monotonic":2406324.05207777,"hcas":{"mlx5_0":{"port_xmit_data":138738688325416,"port_rcv_data":138763973541542,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1837897721499,"port_rcv_data":1192119238210,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138858570776897,"port_rcv_data":139149762171306,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2188606198849,"port_rcv_data":1505261356368,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:29.200","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":198.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:29.218","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":177.71,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:29.231","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":203.15,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:29.244","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":172.11,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:29.255","gpu_util_pct":97.0,"memory_used_mib":64557.0,"power_w":226.79,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:29.266","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":196.12,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:29.278","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":215.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:29.290","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":229.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315450.0565321,"monotonic":2406325.052857767,"hcas":{"mlx5_0":{"port_xmit_data":138738809665340,"port_rcv_data":138763973670550,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838019855223,"port_rcv_data":1192119367108,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138858745327108,"port_rcv_data":139149762455038,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2188780763038,"port_rcv_data":1505261653697,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:30.144","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":265.14,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:30.161","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":256.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:30.172","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":281.29,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:30.183","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":267.05,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:30.194","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":238.53,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:30.204","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":278.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:30.215","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":255.95,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:30.225","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":253.86,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315451.0574002,"monotonic":2406326.053725514,"hcas":{"mlx5_0":{"port_xmit_data":138738809669696,"port_rcv_data":138763973672334,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838019858731,"port_rcv_data":1192119368224,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138858983040344,"port_rcv_data":139149762807140,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189018469211,"port_rcv_data":1505261999625,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:31.203","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":227.23,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:31.221","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":191.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:31.233","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":183.1,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:31.246","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":194.31,"sm_clock_mhz":2295.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:31.258","gpu_util_pct":98.0,"memory_used_mib":64557.0,"power_w":290.1,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:31.271","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":286.99,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:31.282","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":263.93,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:31.293","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":305.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315452.058187,"monotonic":2406327.054511322,"hcas":{"mlx5_0":{"port_xmit_data":138738886255051,"port_rcv_data":138763973748317,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838096443691,"port_rcv_data":1192119443709,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138859013066797,"port_rcv_data":139149762895688,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189048446504,"port_rcv_data":1505262086433,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:32.209","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":163.69,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:32.224","gpu_util_pct":42.0,"memory_used_mib":59345.0,"power_w":173.74,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:32.235","gpu_util_pct":0.0,"memory_used_mib":59345.0,"power_w":171.65,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:32.246","gpu_util_pct":11.0,"memory_used_mib":59345.0,"power_w":168.43,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:32.256","gpu_util_pct":72.0,"memory_used_mib":64557.0,"power_w":150.42,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:32.267","gpu_util_pct":96.0,"memory_used_mib":64557.0,"power_w":147.37,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:32.278","gpu_util_pct":90.0,"memory_used_mib":64557.0,"power_w":145.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:32.288","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":156.55,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315453.0589569,"monotonic":2406328.055284079,"hcas":{"mlx5_0":{"port_xmit_data":138738900496721,"port_rcv_data":138763973772772,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838111478247,"port_rcv_data":1192119468695,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138859187611127,"port_rcv_data":139149763174920,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189223020669,"port_rcv_data":1505262395366,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:33.208","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":247.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:33.226","gpu_util_pct":96.0,"memory_used_mib":59345.0,"power_w":259.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:33.238","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":265.84,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:33.250","gpu_util_pct":94.0,"memory_used_mib":59345.0,"power_w":268.68,"sm_clock_mhz":2310.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:33.261","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":219.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:33.272","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":223.02,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:33.283","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":227.93,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:33.293","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":254.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315454.0597537,"monotonic":2406329.056079395,"hcas":{"mlx5_0":{"port_xmit_data":138738977086756,"port_rcv_data":138763973851481,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838188067887,"port_rcv_data":1192119546636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138859263924381,"port_rcv_data":139149763350227,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189299331531,"port_rcv_data":1505262577380,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:34.189","gpu_util_pct":94.0,"memory_used_mib":59345.0,"power_w":242.03,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:34.206","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":244.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:34.218","gpu_util_pct":97.0,"memory_used_mib":59345.0,"power_w":233.27,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:34.229","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":249.87,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:34.239","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":300.02,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:34.249","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":305.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:34.259","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":311.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:34.270","gpu_util_pct":97.0,"memory_used_mib":64557.0,"power_w":305.21,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315455.0600457,"monotonic":2406330.05637199,"hcas":{"mlx5_0":{"port_xmit_data":138739057740529,"port_rcv_data":138763973940787,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838269515915,"port_rcv_data":1192119636566,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138859438463153,"port_rcv_data":139149763624148,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189473877359,"port_rcv_data":1505262858611,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:35.213","gpu_util_pct":72.0,"memory_used_mib":59345.0,"power_w":269.38,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:35.229","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":243.17,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:35.240","gpu_util_pct":93.0,"memory_used_mib":59345.0,"power_w":252.2,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:35.251","gpu_util_pct":91.0,"memory_used_mib":59345.0,"power_w":246.15,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:35.262","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":233.99,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:35.272","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":226.4,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:35.282","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":234.88,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:35.293","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":219.65,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315456.0608194,"monotonic":2406331.057145038,"hcas":{"mlx5_0":{"port_xmit_data":138739134329845,"port_rcv_data":138763974019296,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838346104383,"port_rcv_data":1192119714578,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138859613662138,"port_rcv_data":139149763931409,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189649039065,"port_rcv_data":1505263172719,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:36.215","gpu_util_pct":98.0,"memory_used_mib":59345.0,"power_w":216.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:36.234","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":219.4,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:36.247","gpu_util_pct":98.0,"memory_used_mib":59345.0,"power_w":222.81,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:36.263","gpu_util_pct":100.0,"memory_used_mib":59345.0,"power_w":231.54,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:36.276","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":291.03,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:36.291","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":302.54,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:36.305","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":277.16,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:36.319","gpu_util_pct":100.0,"memory_used_mib":64557.0,"power_w":294.77,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315457.0615823,"monotonic":2406332.057908222,"hcas":{"mlx5_0":{"port_xmit_data":138739148689920,"port_rcv_data":138763974035966,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838366595808,"port_rcv_data":1192119735976,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138859644531982,"port_rcv_data":139149763989637,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189683779102,"port_rcv_data":1505263235603,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:37.218","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":206.06,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:37.233","gpu_util_pct":24.0,"memory_used_mib":59349.0,"power_w":181.28,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:37.245","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":228.93,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:37.256","gpu_util_pct":0.0,"memory_used_mib":59349.0,"power_w":205.49,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:37.266","gpu_util_pct":53.0,"memory_used_mib":64557.0,"power_w":184.4,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:37.277","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":180.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:37.287","gpu_util_pct":86.0,"memory_used_mib":64557.0,"power_w":199.53,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:37.298","gpu_util_pct":99.0,"memory_used_mib":64557.0,"power_w":177.02,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315458.0623715,"monotonic":2406333.058695964,"hcas":{"mlx5_0":{"port_xmit_data":138739256866740,"port_rcv_data":138763974150993,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838469434715,"port_rcv_data":1192119847104,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138859962756854,"port_rcv_data":139149764500004,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2189998132944,"port_rcv_data":1505263741403,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:38.206","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:38.222","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":84.49,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:38.233","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":101.86,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:38.244","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":102.99,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:38.254","gpu_util_pct":98.0,"memory_used_mib":64561.0,"power_w":262.52,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:38.263","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":228.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:38.273","gpu_util_pct":71.0,"memory_used_mib":64561.0,"power_w":278.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:38.282","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":227.32,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315459.0631733,"monotonic":2406334.059499731,"hcas":{"mlx5_0":{"port_xmit_data":138739256872140,"port_rcv_data":138763974152901,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838469439395,"port_rcv_data":1192119848344,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860028416039,"port_rcv_data":139149764579583,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190063750191,"port_rcv_data":1505263819731,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:39.216","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":137.25,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:39.232","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.35,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:39.243","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.9,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:39.253","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":82.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:39.263","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.68,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:39.272","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.55,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:39.282","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":90.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:39.291","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":85.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315460.0639713,"monotonic":2406335.060297557,"hcas":{"mlx5_0":{"port_xmit_data":138739339919259,"port_rcv_data":138763974244741,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838553280639,"port_rcv_data":1192119939654,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860028421861,"port_rcv_data":139149764582133,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190063754935,"port_rcv_data":1505263820971,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:40.209","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.76,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:40.224","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.39,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:40.236","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:40.246","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":82.21,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:40.256","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:40.266","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:40.275","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.38,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:40.284","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.79,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315461.0647564,"monotonic":2406336.06108372,"hcas":{"mlx5_0":{"port_xmit_data":138739339923615,"port_rcv_data":138763974246525,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838553284147,"port_rcv_data":1192119940770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860149032780,"port_rcv_data":139149764710820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190184365347,"port_rcv_data":1505263948009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:41.206","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.86,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:41.219","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.31,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:41.231","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.78,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:41.242","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":82.02,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:41.252","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.68,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:41.262","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.79,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:41.272","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.33,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:41.282","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.75,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315462.0655444,"monotonic":2406337.061872088,"hcas":{"mlx5_0":{"port_xmit_data":138739339923765,"port_rcv_data":138763974246636,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838553284147,"port_rcv_data":1192119940770,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860149032780,"port_rcv_data":139149764710820,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190184365347,"port_rcv_data":1505263948009,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:42.216","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:42.231","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.21,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:42.242","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:42.253","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.72,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:42.263","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:42.273","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.62,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:42.282","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.14,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:42.292","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.57,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315463.0663257,"monotonic":2406338.062653441,"hcas":{"mlx5_0":{"port_xmit_data":138739421779396,"port_rcv_data":138763974340588,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838635933463,"port_rcv_data":1192120033617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860267852475,"port_rcv_data":139149764838341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190303184419,"port_rcv_data":1505264073962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:43.221","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.7,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:43.234","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:43.245","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.49,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:43.256","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.96,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:43.266","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.67,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:43.276","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:43.285","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.28,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:43.294","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.54,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315464.067104,"monotonic":2406339.063431593,"hcas":{"mlx5_0":{"port_xmit_data":138739421779611,"port_rcv_data":138763974340764,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838635933463,"port_rcv_data":1192120033617,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860267852475,"port_rcv_data":139149764838341,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190303184419,"port_rcv_data":1505264073962,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:44.210","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.65,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:44.224","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.2,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:44.235","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.3,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:44.246","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.81,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:44.256","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.19,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:44.266","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.54,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:44.275","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:44.286","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.35,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315465.0677226,"monotonic":2406340.064049806,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:45.219","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.72,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:45.232","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.14,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:45.245","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:45.256","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.65,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:45.267","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.49,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:45.278","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.57,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:45.288","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.96,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:45.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.27,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315466.0684962,"monotonic":2406341.064823668,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:46.194","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.57,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:46.210","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.14,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:46.222","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:46.234","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:46.247","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.45,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:46.257","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:46.267","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:46.279","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.2,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315467.0692885,"monotonic":2406342.06561607,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:47.227","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.15,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:47.239","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.15,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:47.250","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:47.261","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.68,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:47.271","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:47.280","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:47.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.62,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:47.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315468.0700724,"monotonic":2406343.06639993,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:48.222","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.66,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:48.236","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.69,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:48.247","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:48.257","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.4,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:48.267","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.82,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:48.277","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.72,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:48.286","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.9,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:48.295","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.48,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315469.0708585,"monotonic":2406344.067185941,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:49.225","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:49.238","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.88,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:49.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.8,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:49.260","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.36,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:49.270","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:49.280","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.91,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:49.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.25,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:49.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.95,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315470.0716422,"monotonic":2406345.067969213,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:50.222","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:50.236","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.5,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:50.247","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:50.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.69,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:50.268","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:50.277","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:50.287","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:50.296","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.59,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315471.0724285,"monotonic":2406346.068755942,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:51.227","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:51.243","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.46,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:51.256","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.32,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:51.268","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:51.279","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:51.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:51.300","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:51.309","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.46,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315472.073188,"monotonic":2406347.069517006,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:52.222","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:52.237","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.06,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:52.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.22,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:52.260","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.74,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:52.271","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.74,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:52.280","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:52.290","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.79,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:52.299","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.26,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315473.0739882,"monotonic":2406348.070313278,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:53.255","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:53.271","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.32,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:53.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.05,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:53.292","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.42,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:53.301","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.86,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:53.310","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.19,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:53.319","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:53.328","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.89,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315474.0747576,"monotonic":2406349.071083895,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:54.217","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:54.233","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.46,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:54.244","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.16,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:54.255","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:54.264","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:54.274","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.5,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:54.283","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.18,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:54.293","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.07,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315475.0755637,"monotonic":2406350.071889968,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:55.227","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:55.248","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.22,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:55.268","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:55.290","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:55.313","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:55.335","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:55.346","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:55.357","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.22,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315476.0762954,"monotonic":2406351.072621494,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:56.221","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:56.235","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:56.246","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.08,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:56.256","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.12,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:56.266","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:56.275","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:56.284","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:56.293","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.02,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315477.077111,"monotonic":2406352.073435613,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:57.225","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:57.240","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.57,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:57.251","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:57.261","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.18,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:57.271","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.21,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:57.281","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.98,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:57.291","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.87,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:57.300","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.1,"sm_clock_mhz":2347.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315478.0779061,"monotonic":2406353.074233361,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:58.229","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.68,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:58.244","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:58.255","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.75,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:58.265","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.14,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:58.274","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:58.283","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.99,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:58.292","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.46,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:58.301","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.65,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315479.0787015,"monotonic":2406354.075027668,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:17:59.222","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.1,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:17:59.237","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2370.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:17:59.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.11,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:17:59.259","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.66,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:17:59.269","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:17:59.279","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:17:59.288","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:17:59.297","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315480.0795097,"monotonic":2406355.075835793,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:00.224","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.73,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:00.238","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.42,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:00.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.92,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:00.260","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.26,"sm_clock_mhz":2302.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:00.270","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:00.280","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.08,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:00.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:00.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315481.0797284,"monotonic":2406356.076055457,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:01.226","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:01.240","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.34,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:01.251","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.05,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:01.262","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.62,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:01.272","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.29,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:01.281","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.31,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:01.291","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:01.300","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":83.35,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315482.0805268,"monotonic":2406357.076854712,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:02.226","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.98,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:02.239","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:02.252","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:02.267","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.39,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:02.279","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.95,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:02.291","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:02.303","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.62,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:02.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315483.0813081,"monotonic":2406358.077634947,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:03.238","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:03.254","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.02,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:03.265","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.12,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:03.276","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:03.287","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:03.297","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:03.306","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.7,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:03.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":82.02,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315484.0820994,"monotonic":2406359.078425168,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:04.237","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.63,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:04.253","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:04.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:04.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:04.285","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:04.295","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:04.305","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:04.314","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315485.0828896,"monotonic":2406360.079215582,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:05.233","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.56,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:05.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:05.260","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.31,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:05.271","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:05.281","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:05.291","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:05.300","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:05.310","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315486.083713,"monotonic":2406361.080038449,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:06.226","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:06.242","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:06.253","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:06.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:06.274","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:06.284","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:06.294","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:06.303","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315487.0845451,"monotonic":2406362.080869246,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:07.237","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:07.252","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:07.263","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.02,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:07.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:07.283","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:07.293","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:07.302","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:07.311","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315488.0853448,"monotonic":2406363.08167116,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:08.234","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:08.247","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:08.259","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:08.269","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:08.279","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:08.288","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:08.297","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:08.306","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315489.0861475,"monotonic":2406364.082473576,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:09.239","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:09.257","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:09.270","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:09.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:09.290","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:09.300","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:09.310","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:09.319","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315490.0869415,"monotonic":2406365.083266974,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:10.246","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:10.261","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:10.272","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:10.282","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:10.291","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:10.303","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:10.316","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:10.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.97,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315491.0877328,"monotonic":2406366.084058766,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:11.237","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.23,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:11.252","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:11.263","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:11.274","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:11.284","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:11.293","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:11.302","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:11.311","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.73,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315492.0885406,"monotonic":2406367.084866183,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:12.242","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:12.257","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:12.268","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:12.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:12.288","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:12.297","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:12.306","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:12.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315493.0893443,"monotonic":2406368.085669422,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:13.241","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:13.257","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:13.268","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:13.279","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:13.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:13.299","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:13.308","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:13.318","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315494.0901532,"monotonic":2406369.08647882,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:14.239","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:14.253","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:14.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:14.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:14.285","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:14.295","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:14.305","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:14.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315495.0909402,"monotonic":2406370.087267414,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:15.238","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:15.253","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.73,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:15.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:15.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:15.285","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:15.294","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:15.304","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:15.312","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315496.0917473,"monotonic":2406371.088073655,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:16.242","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:16.259","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:16.272","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:16.284","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:16.296","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:16.307","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:16.318","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:16.329","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315497.092534,"monotonic":2406372.088859494,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:17.243","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:17.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:17.270","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:17.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:17.291","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:17.301","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:17.310","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:17.319","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.51,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315498.0933309,"monotonic":2406373.089657382,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:18.250","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:18.263","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:18.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:18.285","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.36,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:18.295","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:18.305","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:18.314","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:18.323","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.53,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315499.0936248,"monotonic":2406374.089952456,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:19.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:19.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:19.277","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:19.288","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:19.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:19.308","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:19.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:19.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315500.09442,"monotonic":2406375.090747659,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:20.247","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.79,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:20.260","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:20.271","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:20.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:20.292","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:20.304","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:20.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:20.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315501.0951931,"monotonic":2406376.091521135,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:21.238","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.49,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:21.252","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:21.263","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:21.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:21.285","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:21.296","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.17,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:21.305","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:21.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315502.0957272,"monotonic":2406377.09205483,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:22.246","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.47,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:22.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:22.269","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:22.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:22.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:22.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:22.307","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.58,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:22.316","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315503.096513,"monotonic":2406378.092840748,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:23.242","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:23.256","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:23.267","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:23.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.12,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:23.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:23.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:23.308","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:23.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315504.0972984,"monotonic":2406379.093626529,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:24.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.3,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:24.261","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.73,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:24.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:24.284","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":81.14,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:24.294","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:24.303","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:24.312","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:24.322","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315505.098076,"monotonic":2406380.094403957,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:25.251","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.34,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:25.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:25.276","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.59,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:25.287","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:25.297","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:25.307","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:25.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:25.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315506.0984118,"monotonic":2406381.094738984,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:26.244","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:26.257","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.46,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:26.270","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.35,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:26.282","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:26.294","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:26.306","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:26.316","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:26.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315507.0991879,"monotonic":2406382.095515898,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:27.250","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:27.267","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:27.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:27.293","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:27.304","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:27.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:27.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:27.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315508.0997493,"monotonic":2406383.096074116,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:28.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:28.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:28.302","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:28.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:28.324","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:28.335","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:28.345","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:28.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315509.1004937,"monotonic":2406384.096820522,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:29.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.03,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:29.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:29.285","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:29.296","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:29.307","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:29.316","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:29.325","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:29.334","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315510.1012716,"monotonic":2406385.097599365,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:30.186","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:30.200","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:30.211","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:30.222","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.56,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:30.233","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:30.243","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:30.252","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:30.262","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315511.1021223,"monotonic":2406386.098450097,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:31.249","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.89,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:31.262","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:31.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:31.284","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:31.294","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.15,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:31.304","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:31.314","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:31.323","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.05,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315512.1029065,"monotonic":2406387.099234184,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:32.250","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:32.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:32.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:32.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:32.296","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:32.305","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:32.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:32.324","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.75,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315513.1037204,"monotonic":2406388.100048182,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:33.250","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:33.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:33.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:33.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:33.297","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:33.306","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:33.316","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:33.325","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315514.10451,"monotonic":2406389.100837989,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:34.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:34.271","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:34.283","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:34.293","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:34.303","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:34.312","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:34.321","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:34.330","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315515.1052973,"monotonic":2406390.101624888,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:35.256","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:35.269","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:35.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:35.290","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:35.300","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:35.309","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:35.318","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:35.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315516.1060865,"monotonic":2406391.102414039,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:36.257","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:36.270","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:36.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:36.291","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:36.301","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:36.310","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:36.320","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:36.329","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.24,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315517.106873,"monotonic":2406392.103200853,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:37.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:37.271","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:37.282","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:37.292","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.88,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:37.302","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:37.312","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:37.321","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:37.330","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315518.1076663,"monotonic":2406393.103993871,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:38.262","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:38.276","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:38.287","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.8,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:38.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:38.307","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:38.316","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:38.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:38.335","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315519.1077182,"monotonic":2406394.104045006,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:39.263","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:39.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:39.293","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:39.305","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:39.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:39.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:39.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:39.347","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315520.1084971,"monotonic":2406395.10482236,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:40.262","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:40.279","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:40.292","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:40.305","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:40.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:40.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:40.337","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:40.349","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315521.1092808,"monotonic":2406396.105605676,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:41.266","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.58,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:41.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:41.291","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:41.302","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:41.312","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:41.321","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:41.330","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:41.339","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.71,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315522.110069,"monotonic":2406397.106396039,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:42.242","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:42.257","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:42.268","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:42.279","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:42.289","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:42.298","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:42.308","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:42.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315523.1108902,"monotonic":2406398.107216064,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:43.262","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:43.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:43.291","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:43.301","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:43.310","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:43.320","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:43.330","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:43.340","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315524.1117055,"monotonic":2406399.108031135,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:44.268","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:44.282","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:44.294","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:44.304","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:44.314","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:44.324","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:44.333","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:44.342","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315525.112488,"monotonic":2406400.10881479,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:45.265","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:45.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:45.290","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:45.300","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:45.311","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:45.320","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:45.330","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:45.339","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315526.1132734,"monotonic":2406401.109601046,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:46.258","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:46.272","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:46.283","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:46.294","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:46.304","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:46.314","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:46.323","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:46.332","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315527.1140628,"monotonic":2406402.110390195,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:47.266","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:47.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:47.289","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:47.300","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:47.310","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:47.320","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:47.329","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:47.338","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315528.1148462,"monotonic":2406403.111173208,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:48.262","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:48.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:48.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:48.296","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:48.306","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:48.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:48.324","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:48.333","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315529.1150067,"monotonic":2406404.111334189,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:49.262","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:49.275","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.51,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:49.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:49.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:49.308","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:49.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:49.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:49.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315530.11571,"monotonic":2406405.112038189,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:50.263","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:50.276","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:50.287","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:50.298","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:50.308","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:50.318","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:50.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:50.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315531.1164463,"monotonic":2406406.11277378,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:51.269","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:51.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:51.292","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:51.302","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:51.312","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:51.321","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:51.330","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:51.339","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315532.1166644,"monotonic":2406407.112992381,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:52.261","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:52.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:52.284","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.07,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:52.295","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:52.304","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:52.313","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:52.322","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:52.331","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315533.1174598,"monotonic":2406408.11378753,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:53.274","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:53.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:53.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:53.308","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:53.317","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.17,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:53.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:53.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:53.345","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315534.118245,"monotonic":2406409.114572561,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:54.261","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:54.274","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:54.285","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:54.296","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:54.307","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:54.316","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:54.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:54.335","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315535.1190345,"monotonic":2406410.115362078,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:55.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:55.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:55.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:55.308","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:55.318","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:55.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:55.337","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:55.346","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315536.1197276,"monotonic":2406411.116054933,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:56.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:56.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:56.289","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:56.299","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:56.309","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:56.319","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:56.328","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:56.337","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.0,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315537.120517,"monotonic":2406412.116844831,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:57.268","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:57.285","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:57.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:57.310","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:57.320","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:57.329","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:57.338","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:57.347","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315538.121325,"monotonic":2406413.117650059,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:58.270","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":133.55,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:58.284","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:58.295","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:58.306","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:58.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:58.325","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:58.334","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":79.05,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:58.343","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315539.122111,"monotonic":2406414.118438083,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:18:59.261","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:18:59.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.97,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:18:59.285","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:18:59.295","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:18:59.306","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:18:59.315","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:18:59.324","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:18:59.333","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.49,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315540.1229138,"monotonic":2406415.119240604,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:00.274","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:00.287","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:00.299","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.29,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:00.309","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.09,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:00.319","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:00.328","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:00.337","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.1,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:00.346","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315541.1237073,"monotonic":2406416.120035039,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:01.301","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:01.314","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:01.326","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:01.337","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:01.347","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:01.357","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:01.367","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:01.376","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315542.1244621,"monotonic":2406417.12078989,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:02.283","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:02.296","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:02.307","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:02.318","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:02.329","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:02.339","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:02.348","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:02.358","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315543.1252496,"monotonic":2406418.121577367,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:03.289","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:03.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:03.331","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:03.342","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:03.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:03.364","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:03.374","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:03.384","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315544.125995,"monotonic":2406419.122322912,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:04.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:04.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:04.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:04.308","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:04.318","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:04.328","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:04.338","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:04.347","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315545.1267817,"monotonic":2406420.123109677,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:05.273","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:05.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:05.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:05.308","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:05.318","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:05.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:05.342","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:05.360","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315546.1275208,"monotonic":2406421.1238484,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:06.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:06.291","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:06.302","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:06.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:06.323","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:06.332","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:06.341","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.32,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:06.350","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315547.1277156,"monotonic":2406422.12404262,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:07.277","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:07.292","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:07.304","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.54,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:07.315","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:07.325","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:07.335","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:07.345","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:07.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315548.1278048,"monotonic":2406423.124132028,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:08.285","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.99,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:08.300","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:08.311","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.56,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:08.321","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:08.331","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:08.341","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:08.350","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:08.359","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315549.1286092,"monotonic":2406424.124934275,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:09.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:09.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:09.307","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.32,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:09.318","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:09.327","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:09.337","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:09.346","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:09.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315550.129419,"monotonic":2406425.125744828,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:10.281","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:10.296","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.51,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:10.308","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:10.320","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:10.332","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:10.343","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:10.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:10.364","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315551.1302013,"monotonic":2406426.126526753,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:11.282","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:11.297","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:11.308","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:11.319","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:11.329","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":76.9,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:11.339","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:11.348","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:11.357","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315552.1309886,"monotonic":2406427.127315938,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:12.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:12.299","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:12.310","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:12.321","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:12.331","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:12.341","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:12.350","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:12.359","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315553.1317222,"monotonic":2406428.128050051,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:13.279","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:13.292","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:13.303","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:13.314","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:13.324","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:13.334","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:13.343","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:13.352","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315554.132511,"monotonic":2406429.128838396,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:14.277","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.74,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:14.290","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:14.301","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:14.312","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:14.323","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":76.66,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:14.332","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:14.342","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:14.351","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315555.1332994,"monotonic":2406430.129627401,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:15.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:15.291","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:15.302","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:15.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:15.324","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:15.333","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:15.343","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:15.352","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315556.134086,"monotonic":2406431.130413658,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:16.278","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:16.292","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:16.303","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:16.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:16.323","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:16.332","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:16.342","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:16.351","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315557.1348984,"monotonic":2406432.13122379,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:17.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:17.301","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:17.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:17.324","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:17.334","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:17.344","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:17.353","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:17.363","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315558.1357358,"monotonic":2406433.132061051,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:18.283","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.04,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:18.298","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:18.310","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:18.323","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:18.334","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:18.344","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:18.356","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:18.366","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315559.1365378,"monotonic":2406434.132863158,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:19.279","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.52,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:19.294","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:19.306","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:19.316","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:19.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:19.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:19.345","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.61,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:19.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315560.1368728,"monotonic":2406435.133197747,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:20.287","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:20.302","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:20.312","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.34,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:20.323","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:20.332","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:20.341","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:20.350","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.36,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:20.359","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315561.1376615,"monotonic":2406436.133988356,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:21.283","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:21.298","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.75,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:21.309","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:21.319","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:21.329","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:21.338","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:21.347","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:21.356","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.9,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315562.1384947,"monotonic":2406437.134820372,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:22.280","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:22.294","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:22.306","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:22.316","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:22.326","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:22.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:22.345","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:22.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.39,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315563.139335,"monotonic":2406438.135658305,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:23.294","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:23.309","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:23.320","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:23.330","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:23.340","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:23.349","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:23.359","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.34,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:23.367","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.14,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315564.1401644,"monotonic":2406439.13648839,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:24.296","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:24.310","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:24.321","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:24.332","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:24.342","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:24.352","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2362.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:24.363","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:24.374","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315565.1405537,"monotonic":2406440.136880898,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:25.294","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:25.307","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:25.317","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:25.328","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:25.338","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:25.347","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:25.356","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.07,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:25.365","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315566.1413438,"monotonic":2406441.137670324,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:26.295","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.77,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:26.307","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:26.319","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:26.329","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:26.339","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:26.349","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:26.358","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:26.367","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315567.142126,"monotonic":2406442.138453786,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:27.294","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:27.307","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:27.319","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:27.330","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:27.340","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:27.349","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:27.358","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.59,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:27.368","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315568.1429048,"monotonic":2406443.139232409,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:28.290","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.53,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:28.303","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:28.314","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:28.325","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:28.335","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:28.345","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:28.354","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:28.363","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315569.143711,"monotonic":2406444.140038946,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:29.293","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:29.306","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:29.318","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:29.329","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:29.339","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:29.349","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:29.358","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:29.368","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315570.1445003,"monotonic":2406445.1408283,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:30.229","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.54,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:30.241","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:30.253","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.61,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:30.264","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":80.07,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:30.274","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.64,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:30.283","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:30.293","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:30.302","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.88,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315571.1453557,"monotonic":2406446.141683418,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:31.290","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":136.07,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:31.305","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:31.316","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:31.326","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.83,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:31.336","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.02,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:31.346","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:31.355","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:31.364","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":81.41,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315572.1461515,"monotonic":2406447.1424792,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:32.286","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.5,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:32.299","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:32.310","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:32.321","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:32.331","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.83,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:32.341","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":76.95,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:32.350","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:32.359","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315573.1469529,"monotonic":2406448.143280366,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:33.302","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.45,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:33.317","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.49,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:33.328","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:33.338","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.61,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:33.347","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:33.357","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.93,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:33.366","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:33.375","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.66,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315574.1477451,"monotonic":2406449.144069809,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:34.319","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.01,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:34.334","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.24,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:34.346","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:34.356","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:34.366","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.39,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:34.376","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:34.386","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":78.12,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:34.395","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.68,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315575.1485426,"monotonic":2406450.144866322,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:35.301","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":135.26,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:35.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.75,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:35.323","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:35.332","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:35.341","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:35.349","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:35.358","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.88,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:35.367","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315576.1493642,"monotonic":2406451.145689481,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:36.325","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":133.94,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:36.351","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2377.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:36.369","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":78.85,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:36.380","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:36.390","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.14,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:36.400","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.44,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:36.410","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.63,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:36.419","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}
+{"wall_time":1788315577.1501346,"monotonic":2406452.146459549,"hcas":{"mlx5_0":{"port_xmit_data":138739430042812,"port_rcv_data":138763974360036,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_1":{"port_xmit_data":1838644989403,"port_rcv_data":1192120053131,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_2":{"port_xmit_data":138860276281570,"port_rcv_data":139149764857989,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0},"mlx5_3":{"port_xmit_data":2190311613267,"port_rcv_data":1505264092701,"port_xmit_wait":0,"port_rcv_errors":0,"port_xmit_discards":0}},"gpus":[{"index":0,"timestamp":"2026/09/02 10:19:37.298","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":134.48,"sm_clock_mhz":2340.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":1,"timestamp":"2026/09/02 10:19:37.313","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.0,"sm_clock_mhz":2385.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":2,"timestamp":"2026/09/02 10:19:37.324","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.1,"sm_clock_mhz":2325.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":3,"timestamp":"2026/09/02 10:19:37.334","gpu_util_pct":0.0,"memory_used_mib":59351.0,"power_w":79.58,"sm_clock_mhz":2317.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":4,"timestamp":"2026/09/02 10:19:37.343","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":76.56,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":5,"timestamp":"2026/09/02 10:19:37.353","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.19,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":6,"timestamp":"2026/09/02 10:19:37.362","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":77.85,"sm_clock_mhz":2332.0,"pcie_gen":5.0,"pcie_width":16.0},{"index":7,"timestamp":"2026/09/02 10:19:37.370","gpu_util_pct":0.0,"memory_used_mib":64561.0,"power_w":80.42,"sm_clock_mhz":2355.0,"pcie_gen":5.0,"pcie_width":16.0}]}